-- Logs begin at Tue 2022-10-04 08:10:17 UTC, end at Tue 2022-10-04 09:16:13 UTC. -- Oct 04 08:37:01.014430 dv-cinder-844563-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 08:37:01.025505 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 08:37:01 2022] *** Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: nodename: dv-cinder-844563-14-iscsi Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: machine: x86_64 Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: clock source: unix Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: pcre jit disabled Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: detected number of CPU cores: 4 Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: current working directory: / Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: detected binary path: /usr/bin/uwsgi-core Oct 04 08:37:01.027559 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: your processes number limit is 47680 Oct 04 08:37:01.028212 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: your memory page size is 4096 bytes Oct 04 08:37:01.028212 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: detected max file descriptor number: 2048 Oct 04 08:37:01.028212 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: lock engine: pthread robust mutexes Oct 04 08:37:01.028212 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: thunder lock: enabled Oct 04 08:37:01.028212 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 08:37:01.028212 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 08:37:01.066383 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: Python main interpreter initialized at 0x558dc77db9c0 Oct 04 08:37:01.066383 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: python threads support enabled Oct 04 08:37:01.066383 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: your server socket listen backlog is limited to 100 connections Oct 04 08:37:01.066383 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: your mercy for graceful operations on workers is 90 seconds Oct 04 08:37:01.066732 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 08:37:01.066807 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: *** Operational MODE: preforking *** Oct 04 08:37:01.067051 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: *** uWSGI is running in multiple interpreter mode *** Oct 04 08:37:01.067051 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: spawned uWSGI master process (pid: 103414) Oct 04 08:37:01.067249 dv-cinder-844563-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 08:37:01.068067 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: spawned uWSGI worker 1 (pid: 103426, cores: 1) Oct 04 08:37:01.068067 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: spawned uWSGI worker 2 (pid: 103427, cores: 1) Oct 04 08:37:01.068242 dv-cinder-844563-14-iscsi devstack@keystone.service[103414]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 08:37:05.271841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:37:05.271841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: from cryptography.utils import int_from_bytes Oct 04 08:37:05.272653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:37:05.272653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: from cryptography.utils import int_from_bytes Oct 04 08:37:05.272776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:37:05.272776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: from cryptography.utils import int_from_bytes Oct 04 08:37:05.273181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:37:05.273181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: from cryptography.utils import int_from_bytes Oct 04 08:37:05.321024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:37:05.321277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:37:05.321487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] command line args: [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:37:05.321626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:37:05.321705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:37:05.321799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:37:05.321872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ================================================================================ {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:37:05.322003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] command line args: [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:37:05.322145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] admin_token = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.322229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:37:05.322303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] config_dir = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.322383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ================================================================================ {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:37:05.322456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.322662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] config_source = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.322798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] admin_token = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.322881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] debug = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.322950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] config_dir = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] config_source = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] debug = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] insecure_debug = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.323978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] insecure_debug = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] list_limit = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] log_config_append = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] list_limit = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] log_dir = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] log_config_append = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.324915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] log_file = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] log_dir = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] log_file = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.325975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.326918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] notification_format = cadf {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.327952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.328148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.328273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.328366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] notification_format = cadf {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.328629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.328766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.328862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] publish_errors = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] publish_errors = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.329977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] standard_threads = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330261 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] strict_password_check = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] standard_threads = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] use_eventlog = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] strict_password_check = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] use_journal = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.330969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] use_json = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] use_stderr = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] use_eventlog = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] use_syslog = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] use_journal = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] use_json = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] watch_log_file = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] use_stderr = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.331857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] use_syslog = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.331986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.332081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] watch_log_file = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:37:05.332241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.332325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.332473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.332555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.332714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.332801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.332943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.333959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.334885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.335997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336080 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.336945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.337960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.338875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.339948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.340943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.341991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342720 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.342954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.343992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.344993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.345887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.346940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.347981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.348905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.enabled = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.enabled = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.349980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.350970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.351947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.352942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353272 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.353952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.354932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355788 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.355873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] auth.external = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] auth.mapped = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.356989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] auth.external = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] auth.mapped = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] auth.password = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] auth.token = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.357974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] auth.password = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] catalog.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] auth.token = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] catalog.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.358905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] credential.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] credential.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.359988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] credential.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] credential.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.360982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.361925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362407 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.362906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.363854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.364971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.365997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.366914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.367830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.368934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.369959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.370896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.371974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.372068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.372525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.372615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.372762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.372841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.372985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.373926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.password = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.374912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.password = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.375833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.376901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.377976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.378971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.379926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380080 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.380987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.381988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.382917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383813 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.383957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.384864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] policy.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] policy.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] receipt.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.385927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] receipt.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.386923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.387857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] revoke.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.388965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] revoke.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] role.cache_time = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] role.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] role.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] role.cache_time = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] role.list_limit = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] role.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] role.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] role.list_limit = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.390891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.391849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392793 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.392952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.393032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.393199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.393347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.393435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.393582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.393822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.393902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.394849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.395952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.396968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.cache_time = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397650 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.397883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.cache_time = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.provider = fernet {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.provider = fernet {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.398958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.399962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] trust.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] trust.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.400967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.401939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.connection = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.connection = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.402878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.403972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.404994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.405910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.406065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.406144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.406294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.406381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.406455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:37:05.406527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.406750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:37:05.406920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:37:05.435277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.435503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.435588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.435733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.435866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.435977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.443953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.444299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.471581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.471762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.472134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.472341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.524161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.524666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.571572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.571768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.572452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.572658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.726652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.726856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.726940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.727954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.728043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.750748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.750953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.751837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.752044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.752190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.752263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.763107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.763370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.775747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.775878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776360 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.776935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.777077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.777278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.777464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:37:05.793455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.793645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:37:05.793869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.794036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.794143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:37:05.794274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:37:05.794396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.794516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.794638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:37:05.794775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.794904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.795024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.795181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.795302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.795421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.795564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.804700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.804974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.805202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.805330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.806450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.806556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.806624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.806624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.812539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.813111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.815437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.815717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.816018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.816142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.816267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.816408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.816547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.816634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.816806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.816896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.817996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.818148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.818274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.818388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.818669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.840847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.841587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.842292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.842572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.843095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.843389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.847252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.847532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.847814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.848180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.848465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.848745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.854242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.854522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.855247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.855533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.860192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.860487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.860776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.861058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.861219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.861307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.861514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.861624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.861805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.861949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.862062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.862160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.862363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.862671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.862921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.863188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.877951 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.878227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.879067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.879341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.882088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.883195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.885383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.885680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.886460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.886755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.889911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.890960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.893106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.894117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.896322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.896628 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.896906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.897166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.897354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.897459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.897658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.897749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.897941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.898224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.898500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.898798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.913606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.913888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.914149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.914413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.914797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.915137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.915405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.915666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.921323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.922552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.922846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.924110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.924437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.924709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.925641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.925918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.927405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.927693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.927997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.928282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.928586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.928673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.928876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.929161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.929443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.929843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.939837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.940137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.940412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.940696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.941071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.941164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.941368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.941451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.941645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.941737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.941923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.942311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.942603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.942890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.957275 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.958446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.960183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.961402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.963097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.964371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.964651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.967785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.967981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.967981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.967981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.968413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.968413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.968413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.973128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.974413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.974595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.975873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.977527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.977923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.978205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.978480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.978859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.979255 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.979541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.979815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.986291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.986577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.986858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.987677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.987988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.988285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.993796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=103426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.994077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.994352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.994624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.994898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.995175 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.995299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=103427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:37:05.995478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.995618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.995749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.995875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.996029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.996124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.996250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.996334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.996503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.996636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.996838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.997024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.997465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.997595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.997725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=103426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.997851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.997989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.998243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.998513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.998773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:05.999055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=103427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:37:06.031720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558dc77db9c0 pid: 103426 (default app) Oct 04 08:37:06.033402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558dc77db9c0 pid: 103427 (default app) Oct 04 08:37:06.035039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ace55-3341-4475-9003-36152f0537cb None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:06.035326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ace55-3341-4475-9003-36152f0537cb None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:06.035602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ace55-3341-4475-9003-36152f0537cb None None] PATH_INFO: `/v3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:06.039742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Tue Oct 4 08:37:06 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:12.862475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7da8b-4a67-47ab-b9e4-3a9017325a38 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:12.863016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7da8b-4a67-47ab-b9e4-3a9017325a38 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:12.863016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7da8b-4a67-47ab-b9e4-3a9017325a38 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:12.867268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:12.874557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:12.875467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:12.875467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:12.947690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: INFO dbcounter [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] Registered counter for database keystone Oct 04 08:37:12.967172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG oslo_db.sqlalchemy.engines [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103426) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:37:12.977422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writer thread running {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:37:13.179351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG passlib.handlers.bcrypt [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=103426) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 08:37:13.179640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG passlib.handlers.bcrypt [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=103426) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 08:37:13.237752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:13.239323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:13.388143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 04 08:37:13.937863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7da8b-4a67-47ab-b9e4-3a9017325a38 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:14.008493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: INFO dbcounter [None req-9ca7da8b-4a67-47ab-b9e4-3a9017325a38 None None] Registered counter for database keystone Oct 04 08:37:14.033455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG oslo_db.sqlalchemy.engines [None req-9ca7da8b-4a67-47ab-b9e4-3a9017325a38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103427) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:37:14.045911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writer thread running {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:37:14.173041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7da8b-4a67-47ab-b9e4-3a9017325a38 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:14.178479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee32705-3e4d-4478-bcec-6f289132f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:14.179113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32705-3e4d-4478-bcec-6f289132f8ab admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:14.179396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32705-3e4d-4478-bcec-6f289132f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:14.179658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32705-3e4d-4478-bcec-6f289132f8ab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:14.192702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee32705-3e4d-4478-bcec-6f289132f8ab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:14.244223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bee32705-3e4d-4478-bcec-6f289132f8ab admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 08:37:14.244542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee32705-3e4d-4478-bcec-6f289132f8ab admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:14.255428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Tue Oct 4 08:37:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:37:14.262579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:14.290932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49797ad2-7239-4980-9e3a-dcea9f8bf642 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:14.296741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e51d1-16f5-483d-91c6-ad22e242bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:14.297853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e51d1-16f5-483d-91c6-ad22e242bafd admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:14.298129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e51d1-16f5-483d-91c6-ad22e242bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:14.298398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e51d1-16f5-483d-91c6-ad22e242bafd admin admin] PATH_INFO: `/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:14.300173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e51d1-16f5-483d-91c6-ad22e242bafd admin admin] RBAC: Authorizing `identity:get_project(project_id=5d6b22621efa4e2c8d25ad2792cb5242)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:14.352309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d39e51d1-16f5-483d-91c6-ad22e242bafd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 08:37:14.352651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e51d1-16f5-483d-91c6-ad22e242bafd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:14.357169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Tue Oct 4 08:37:14 2022] GET /identity/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242 => generated 344 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:16.450365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432fef-6630-4ef2-acc1-8d08a8a071da None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:16.451247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432fef-6630-4ef2-acc1-8d08a8a071da None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:16.451247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432fef-6630-4ef2-acc1-8d08a8a071da None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:16.457906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:16.463710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f6757-8bc6-41b8-97c0-ffa1901133e5 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:16.464173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f6757-8bc6-41b8-97c0-ffa1901133e5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:16.464276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f6757-8bc6-41b8-97c0-ffa1901133e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:16.485898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8a8f6757-8bc6-41b8-97c0-ffa1901133e5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:16.487407 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8a8f6757-8bc6-41b8-97c0-ffa1901133e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:16.495814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 08:37:16.760459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432fef-6630-4ef2-acc1-8d08a8a071da None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:16.786228 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432fef-6630-4ef2-acc1-8d08a8a071da None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:16.793340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa93dc9-ca28-4753-87af-2ac91f57a34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:16.793960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93dc9-ca28-4753-87af-2ac91f57a34d admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:16.794518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93dc9-ca28-4753-87af-2ac91f57a34d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:16.794788 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa93dc9-ca28-4753-87af-2ac91f57a34d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:16.803857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa93dc9-ca28-4753-87af-2ac91f57a34d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:16.806041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa93dc9-ca28-4753-87af-2ac91f57a34d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:16.815237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Tue Oct 4 08:37:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:37:16.821194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f6757-8bc6-41b8-97c0-ffa1901133e5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:16.830872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f6757-8bc6-41b8-97c0-ffa1901133e5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:16.836430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfcb04-7448-4ebe-a21b-2f112bd55e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:16.837057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfcb04-7448-4ebe-a21b-2f112bd55e04 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:16.837337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfcb04-7448-4ebe-a21b-2f112bd55e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:16.837608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfcb04-7448-4ebe-a21b-2f112bd55e04 admin admin] PATH_INFO: `/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:16.838966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfcb04-7448-4ebe-a21b-2f112bd55e04 admin admin] RBAC: Authorizing `identity:get_user(user_id=6a7ff5148f6b41108f35cb9aa85b66e2)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:16.840855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfcb04-7448-4ebe-a21b-2f112bd55e04 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:16.842384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Tue Oct 4 08:37:16 2022] GET /identity/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:19.046420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd072f1-a3a4-416b-a9ca-5fe11064b49e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.046963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd072f1-a3a4-416b-a9ca-5fe11064b49e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.046963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd072f1-a3a4-416b-a9ca-5fe11064b49e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.048314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:19.053684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456e8b1-6244-4710-b1a9-6f468dceb52f None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.054135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456e8b1-6244-4710-b1a9-6f468dceb52f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.054224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.054304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456e8b1-6244-4710-b1a9-6f468dceb52f None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.054387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.054659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.055553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:19.065787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96e7f-aec3-4fb3-924b-a177bc5cedc3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.066160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96e7f-aec3-4fb3-924b-a177bc5cedc3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.066502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96e7f-aec3-4fb3-924b-a177bc5cedc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.086534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-91a96e7f-aec3-4fb3-924b-a177bc5cedc3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:19.088036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-91a96e7f-aec3-4fb3-924b-a177bc5cedc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:19.100835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG passlib.handlers.bcrypt [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=103427) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 08:37:19.101234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG passlib.handlers.bcrypt [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=103427) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 08:37:19.165771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:19.167404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:19.172218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 7/12] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:19.272380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 6/13] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:19.432706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a96e7f-aec3-4fb3-924b-a177bc5cedc3 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:19.461285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:19.536388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:19.545302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a96e7f-aec3-4fb3-924b-a177bc5cedc3 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:19.578780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:19.579424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.579705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.580006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.639218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c0add-0a85-4173-a175-ffc8a7670939 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:19.641880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:19.648157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:19.649646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:19.673342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:19.676549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:19.677673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.677956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.678227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.690019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:19.698678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:19.699653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:19.700526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:19.705819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:19.750798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 8/15] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:19.756502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:19.804390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee340c3-dfed-4fa5-8066-2e7e3218a185 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:19.839160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:19.839770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.840080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.840345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.841048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:19.841349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:19.842774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:19.854789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 8/16] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:19.856768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dffb90-13dc-42d2-bb6e-3616d837617a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:19.860302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:19.881230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:19.888643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:19.894380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:19.895102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.895454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.895797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.906714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:19.908227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:37:19.927177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:19.928315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:37:19.929327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:37:19.929327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:37:19.929327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:19.929327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:19.935766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:19.962476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e99144-f9fe-4584-af3a-5a7493211678 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:19.997220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:19.997859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:19.998165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:19.998437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] PATH_INFO: `/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:19.999968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] RBAC: Authorizing `identity:get_user(user_id=6a7ff5148f6b41108f35cb9aa85b66e2)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:20.001423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:20.003005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 9/18] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:19 2022] GET /identity/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:20.008787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:20.036858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd05ff-52a0-4777-8e3f-d51c57abba3a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:20.070886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:20.071475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:20.071702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:20.072009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:20.072668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:20.072968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:20.074380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:20.091870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 10/19] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:20.104882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:20.114453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de188b-59e1-4411-9c82-835cf59ea50f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:20.154620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ac4f7-0730-4963-83af-529442333605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:20.155281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac4f7-0730-4963-83af-529442333605 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:20.155564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac4f7-0730-4963-83af-529442333605 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:20.155836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac4f7-0730-4963-83af-529442333605 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:20.157101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ac4f7-0730-4963-83af-529442333605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:20.158574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ac4f7-0730-4963-83af-529442333605 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:20.160190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 10/20] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:20.166689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:20.204764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:20.278975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1650f-f7de-493e-93cf-6aa27b8197e8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:20.312695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1500c-4701-4759-93a0-8d5c7832be67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:20.313312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1500c-4701-4759-93a0-8d5c7832be67 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:20.313588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1500c-4701-4759-93a0-8d5c7832be67 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:20.313849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1500c-4701-4759-93a0-8d5c7832be67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:20.314583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1500c-4701-4759-93a0-8d5c7832be67 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.domain.id=default, user.id=6a7ff5148f6b41108f35cb9aa85b66e2) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:20.314904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1500c-4701-4759-93a0-8d5c7832be67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:20.316387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1500c-4701-4759-93a0-8d5c7832be67 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:20.323485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 11/21] 10.222.0.44 () {56 vars in 1322 bytes} [Tue Oct 4 08:37:20 2022] GET /identity/v3/role_assignments?user.id=6a7ff5148f6b41108f35cb9aa85b66e2&scope.domain.id=default&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:22.725050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac8d8b-5703-443f-b388-712c59fd6f0f None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:22.725557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac8d8b-5703-443f-b388-712c59fd6f0f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:22.725823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac8d8b-5703-443f-b388-712c59fd6f0f None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:22.727542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 11/22] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:22.733010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99557397-f36c-40a8-b17e-362e1790cab7 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:22.733859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99557397-f36c-40a8-b17e-362e1790cab7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:22.734318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99557397-f36c-40a8-b17e-362e1790cab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:22.760898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-99557397-f36c-40a8-b17e-362e1790cab7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:22.762470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-99557397-f36c-40a8-b17e-362e1790cab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:22.857494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 12/23] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:22.877818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f0294-39bd-4bb4-9b67-07bfb9d6dc5a None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:22.878417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f0294-39bd-4bb4-9b67-07bfb9d6dc5a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:22.878873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f0294-39bd-4bb4-9b67-07bfb9d6dc5a None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:22.880421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 12/24] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:22.886510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24d93-72bd-4f73-b5ed-054561f44a99 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:22.887053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24d93-72bd-4f73-b5ed-054561f44a99 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:22.887674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24d93-72bd-4f73-b5ed-054561f44a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:22.917318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-05a24d93-72bd-4f73-b5ed-054561f44a99 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:22.918898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-05a24d93-72bd-4f73-b5ed-054561f44a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:22.945038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89395-c1e2-4dbf-8043-e7eb1f561aa3 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:22.945774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89395-c1e2-4dbf-8043-e7eb1f561aa3 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:22.946294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89395-c1e2-4dbf-8043-e7eb1f561aa3 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:22.948135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 13/25] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:22.965939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8a366-a1b6-413b-91f3-c7eb2638b2c8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:22.966644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8a366-a1b6-413b-91f3-c7eb2638b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:22.967129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8a366-a1b6-413b-91f3-c7eb2638b2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:22.976129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:23.008237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-87b8a366-a1b6-413b-91f3-c7eb2638b2c8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:23.011226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-87b8a366-a1b6-413b-91f3-c7eb2638b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:23.082179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 13/26] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:23.090381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1fdc8-00a6-4ebd-a5d3-c92edcb84d11 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:23.090670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1fdc8-00a6-4ebd-a5d3-c92edcb84d11 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:23.090950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1fdc8-00a6-4ebd-a5d3-c92edcb84d11 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:23.092305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 14/27] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:23.100871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09ea2-90be-47e4-8808-e2276fac5db0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:23.101155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09ea2-90be-47e4-8808-e2276fac5db0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:23.101483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac09ea2-90be-47e4-8808-e2276fac5db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:23.135380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-cac09ea2-90be-47e4-8808-e2276fac5db0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:23.136625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-cac09ea2-90be-47e4-8808-e2276fac5db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:23.170960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 14/28] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:23.219054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 15/29] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:23.264755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8a366-a1b6-413b-91f3-c7eb2638b2c8 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:23.357950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:23.366815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8a366-a1b6-413b-91f3-c7eb2638b2c8 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:23.402644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:23.403289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:23.403572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:23.403847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:23.414903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:23.419817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:23.420666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:23.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 15/30] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:23.428443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac09ea2-90be-47e4-8808-e2276fac5db0 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:23.543813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:23.557462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac09ea2-90be-47e4-8808-e2276fac5db0 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:23.585585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:23.597436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:23.598340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:23.598830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:23.599305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:23.600214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:23.600721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:23.602532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:23.615284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 16/31] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:23.621118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:23.733678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fa027-7025-45ad-8b0c-6818a0259938 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:23.775658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:23.776582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:23.777014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:23.777524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:23.778252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7557c-31ed-4a36-a4a3-6a8d5deef89b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:23.778825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:23.780588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:23.816873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:23.817610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:23.817993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:23.818409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] PATH_INFO: `/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:23.819687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] RBAC: Authorizing `identity:get_user(user_id=6a7ff5148f6b41108f35cb9aa85b66e2)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:23.821508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:23.823408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 17/32] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:23 2022] GET /identity/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:23.824469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:23.851812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:23.929288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 16/33] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Oct 4 08:37:23 2022] POST /identity/v3/roles => generated 225 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:37:23.930854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:23.959299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932b47e-1b39-49fd-8b37-4a6c904693f6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:23.966022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:24.032789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:24.033672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.034117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.034521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.065139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:24.066963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:37:24.091848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:24.092985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:37:24.093804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:37:24.093804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:37:24.093804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:24.094046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 18/34] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:24.095185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:24.114001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529a829-0080-4f87-b834-75c8aca07bc1 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:24.179537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:24.180479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.180912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.181192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.181945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:24.183469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:24.201641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 17/35] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Oct 4 08:37:23 2022] POST /identity/v3/roles => generated 219 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:24.202497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:24.203518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:24.240297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f3b3b-8ae4-469d-b66a-1b026b3c096b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:24.313905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:24.315364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.316354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.316835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.320264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:24.324776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:24.330045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 19/36] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:24.336318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:24.357753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:24.411545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02467555-bf64-49db-a57d-73ce15853990 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:24.467581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d5b65-a6a2-457c-892b-43295eacbe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:24.468650 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5b65-a6a2-457c-892b-43295eacbe5e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.469214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5b65-a6a2-457c-892b-43295eacbe5e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.469717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5b65-a6a2-457c-892b-43295eacbe5e None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.470605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d5b65-a6a2-457c-892b-43295eacbe5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:24.471122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d5b65-a6a2-457c-892b-43295eacbe5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:24.472749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d5b65-a6a2-457c-892b-43295eacbe5e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:24.492348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e783ef8-5142-43ad-ad50-a724c4905a08 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:24.495913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 18/37] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:24.497811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71e4ef-c9ff-46fd-9017-a628d495daf9 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.498291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71e4ef-c9ff-46fd-9017-a628d495daf9 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.498768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71e4ef-c9ff-46fd-9017-a628d495daf9 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.500194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 19/38] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:24.506161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71e4ef-c9ff-46fd-9017-a628d495daf9 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:24.538177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef68c2a-f775-4c2a-bd89-8d409f6591cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:24.538933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef68c2a-f775-4c2a-bd89-8d409f6591cf None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.539156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef68c2a-f775-4c2a-bd89-8d409f6591cf None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.539429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef68c2a-f775-4c2a-bd89-8d409f6591cf None admin] PATH_INFO: `/v3/domains/default/users/6a7ff5148f6b41108f35cb9aa85b66e2/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.543378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef68c2a-f775-4c2a-bd89-8d409f6591cf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6a7ff5148f6b41108f35cb9aa85b66e2, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:24.545320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef68c2a-f775-4c2a-bd89-8d409f6591cf None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:24.561511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:24.625389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71e4ef-c9ff-46fd-9017-a628d495daf9 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:24.658728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:24.665654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363de52-ae03-4e99-8ae3-37356c732a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:24.666528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363de52-ae03-4e99-8ae3-37356c732a1b None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.666949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363de52-ae03-4e99-8ae3-37356c732a1b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.667399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363de52-ae03-4e99-8ae3-37356c732a1b None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.668274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363de52-ae03-4e99-8ae3-37356c732a1b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:24.669928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363de52-ae03-4e99-8ae3-37356c732a1b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:24.708969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 20/39] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:37:24 2022] POST /identity/v3/projects => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:24.710354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778812a-9231-4e1c-a3b2-82e0479401ad None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.710674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778812a-9231-4e1c-a3b2-82e0479401ad None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.710945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778812a-9231-4e1c-a3b2-82e0479401ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.733806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b778812a-9231-4e1c-a3b2-82e0479401ad None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:24.735454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b778812a-9231-4e1c-a3b2-82e0479401ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:24.815499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 21/40] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:24.821860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 20/41] 10.222.0.44 () {58 vars in 1361 bytes} [Tue Oct 4 08:37:24 2022] PUT /identity/v3/domains/default/users/6a7ff5148f6b41108f35cb9aa85b66e2/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:37:24.832054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f9136-a798-44ef-be61-0c0321766147 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.832054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f9136-a798-44ef-be61-0c0321766147 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.832054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f9136-a798-44ef-be61-0c0321766147 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.832054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 22/42] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:24.838356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be2cec-b216-4cb2-abc6-1e15bf1fe78d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:24.838713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be2cec-b216-4cb2-abc6-1e15bf1fe78d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:24.838933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be2cec-b216-4cb2-abc6-1e15bf1fe78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:24.873366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-17be2cec-b216-4cb2-abc6-1e15bf1fe78d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:24.875081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-17be2cec-b216-4cb2-abc6-1e15bf1fe78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:24.965190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 21/43] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:25.089249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f9136-a798-44ef-be61-0c0321766147 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:25.195675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f9136-a798-44ef-be61-0c0321766147 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:25.209909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:25.232492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:25.233052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:25.233329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:25.233678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:25.234729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be2cec-b216-4cb2-abc6-1e15bf1fe78d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:25.234829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:25.236263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:25.239126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 23/44] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:25.244485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:25.339871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be2cec-b216-4cb2-abc6-1e15bf1fe78d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:25.357653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e919f-8426-4cb1-9044-4baf71fab909 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:25.367283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:25.378124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082356b1-50db-4ada-aecf-d6306ecec5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:25.378759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082356b1-50db-4ada-aecf-d6306ecec5fa None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:25.379042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082356b1-50db-4ada-aecf-d6306ecec5fa None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:25.379314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082356b1-50db-4ada-aecf-d6306ecec5fa None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:25.380035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082356b1-50db-4ada-aecf-d6306ecec5fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:25.381670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082356b1-50db-4ada-aecf-d6306ecec5fa None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:25.403128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 22/45] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Oct 4 08:37:25 2022] POST /identity/v3/roles => generated 223 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:37:25.405627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf9ea-1314-4ecc-8610-cad30fc0a117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:25.406522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf9ea-1314-4ecc-8610-cad30fc0a117 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:25.407011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf9ea-1314-4ecc-8610-cad30fc0a117 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:25.407502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf9ea-1314-4ecc-8610-cad30fc0a117 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:25.408392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf9ea-1314-4ecc-8610-cad30fc0a117 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:25.410185 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf9ea-1314-4ecc-8610-cad30fc0a117 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:25.441267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 24/46] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:37:25 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:26.995872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607b0d8-c7a2-4d41-b5a2-92e7d41d0318 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:26.996432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607b0d8-c7a2-4d41-b5a2-92e7d41d0318 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:26.996432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607b0d8-c7a2-4d41-b5a2-92e7d41d0318 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:26.998095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 23/47] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:27.003681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e62ca-5397-466a-84e6-28e1992742fa None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:27.004121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e62ca-5397-466a-84e6-28e1992742fa None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:27.004277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e62ca-5397-466a-84e6-28e1992742fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:27.023807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-346e62ca-5397-466a-84e6-28e1992742fa None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:27.025043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-346e62ca-5397-466a-84e6-28e1992742fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:27.052940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:27.096985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 25/48] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:27.359714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607b0d8-c7a2-4d41-b5a2-92e7d41d0318 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:27.467554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607b0d8-c7a2-4d41-b5a2-92e7d41d0318 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:27.506538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d02d04-ef1b-4561-8519-7a95f51d4ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:27.507187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02d04-ef1b-4561-8519-7a95f51d4ee3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:27.507494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02d04-ef1b-4561-8519-7a95f51d4ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:27.507764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02d04-ef1b-4561-8519-7a95f51d4ee3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:27.519565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d02d04-ef1b-4561-8519-7a95f51d4ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-a3d02d04-ef1b-4561-8519-7a95f51d4ee3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:27.525863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:27.526830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:27.527597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 24/49] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:27.532963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e62ca-5397-466a-84e6-28e1992742fa None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:27.586309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23c0af-62e8-409b-a4dd-6cc0ed4dde93 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:27.586595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23c0af-62e8-409b-a4dd-6cc0ed4dde93 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:27.586876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23c0af-62e8-409b-a4dd-6cc0ed4dde93 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:27.588299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 25/50] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:27.593205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd66f2-a4ad-45ee-b466-0232df79b58a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:27.593475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd66f2-a4ad-45ee-b466-0232df79b58a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:27.593737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd66f2-a4ad-45ee-b466-0232df79b58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:27.611464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-1ccd66f2-a4ad-45ee-b466-0232df79b58a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:27.612610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1ccd66f2-a4ad-45ee-b466-0232df79b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:27.638691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e62ca-5397-466a-84e6-28e1992742fa None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:27.658885 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:27.673600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:27.674206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:27.674486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:27.674758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:27.675422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:27.675736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:27.677137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:27.682885 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 26/51] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:27.685505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 26/52] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:27.691230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd66f2-a4ad-45ee-b466-0232df79b58a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:27.790438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd66f2-a4ad-45ee-b466-0232df79b58a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:27.823555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:27.824215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:27.824495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:27.824763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] PATH_INFO: `/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:27.825946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] RBAC: Authorizing `identity:get_user(user_id=6a7ff5148f6b41108f35cb9aa85b66e2)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:27.827400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:27.829022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 27/53] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:27 2022] GET /identity/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:27.834569 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:27.863349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:27.935268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699f85e-49ae-4da4-bc2b-b3d31a419289 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:27.941806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:27.997271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:28.056132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53b13b-3341-41a1-af31-d79d0f0b356a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:28.077704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:28.078347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:28.078620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:28.078889 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:28.080028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:28.081681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:28.083443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 27/54] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:28.089516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:28.107668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:28.108338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:28.108599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:28.108868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:28.110013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:28.111469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:28.113115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 28/55] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:28.118695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:28.189387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:28.196682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27968f-2396-4806-a58b-198dc45e7af5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:28.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad899f1-87f9-4921-97a5-f88dc99eca75 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:28.229832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f5d5b-6eca-4866-a6b3-d4ac865a480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:28.230447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f5d5b-6eca-4866-a6b3-d4ac865a480f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:28.230727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f5d5b-6eca-4866-a6b3-d4ac865a480f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:28.231000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f5d5b-6eca-4866-a6b3-d4ac865a480f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:28.231730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f5d5b-6eca-4866-a6b3-d4ac865a480f None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.domain.id=default, user.id=6a7ff5148f6b41108f35cb9aa85b66e2) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:28.232066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f5d5b-6eca-4866-a6b3-d4ac865a480f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:28.232891 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:28.233502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f5d5b-6eca-4866-a6b3-d4ac865a480f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:28.242191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 28/56] 10.222.0.44 () {56 vars in 1322 bytes} [Tue Oct 4 08:37:28 2022] GET /identity/v3/role_assignments?user.id=6a7ff5148f6b41108f35cb9aa85b66e2&scope.domain.id=default&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:28.253584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e036f2-4b7c-4304-ad92-522c172ce195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:28.254221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e036f2-4b7c-4304-ad92-522c172ce195 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:28.254503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e036f2-4b7c-4304-ad92-522c172ce195 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:28.254773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e036f2-4b7c-4304-ad92-522c172ce195 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:28.255441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e036f2-4b7c-4304-ad92-522c172ce195 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:28.257011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e036f2-4b7c-4304-ad92-522c172ce195 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:28.289850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 29/57] 10.222.0.44 () {60 vars in 1140 bytes} [Tue Oct 4 08:37:28 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:37:30.439066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373fa3-417e-4366-8287-4eba4cf8e8df None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:30.439610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373fa3-417e-4366-8287-4eba4cf8e8df None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:30.439610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373fa3-417e-4366-8287-4eba4cf8e8df None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:30.440984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 29/58] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:30.447304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51cbd-f2af-4bb0-815b-5898953ff218 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:30.448029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51cbd-f2af-4bb0-815b-5898953ff218 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:30.448265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51cbd-f2af-4bb0-815b-5898953ff218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:30.472373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b7a51cbd-f2af-4bb0-815b-5898953ff218 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:30.473584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b7a51cbd-f2af-4bb0-815b-5898953ff218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:30.548050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 30/59] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:30.816458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373fa3-417e-4366-8287-4eba4cf8e8df None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:30.931583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373fa3-417e-4366-8287-4eba4cf8e8df None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:30.968826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4ac6c-6e3e-4af2-9df2-fbca9e54b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:30.969456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4ac6c-6e3e-4af2-9df2-fbca9e54b3bb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:30.969741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4ac6c-6e3e-4af2-9df2-fbca9e54b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:30.970373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4ac6c-6e3e-4af2-9df2-fbca9e54b3bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:30.971181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4ac6c-6e3e-4af2-9df2-fbca9e54b3bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:30.972676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4ac6c-6e3e-4af2-9df2-fbca9e54b3bb None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:30.974241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 30/60] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:30.980088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a51cbd-f2af-4bb0-815b-5898953ff218 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:31.078170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:31.084352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a51cbd-f2af-4bb0-815b-5898953ff218 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:31.121066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1529db4-d3ca-423f-88af-45aa3bf00f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:31.121976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529db4-d3ca-423f-88af-45aa3bf00f49 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:31.122253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529db4-d3ca-423f-88af-45aa3bf00f49 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:31.122522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1529db4-d3ca-423f-88af-45aa3bf00f49 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:31.123167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1529db4-d3ca-423f-88af-45aa3bf00f49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:31.124659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1529db4-d3ca-423f-88af-45aa3bf00f49 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:31.152939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 31/61] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:37:30 2022] POST /identity/v3/users => generated 275 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:34.466958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bde7b0-aba9-472d-9a77-5dac17d8469a None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:34.468009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bde7b0-aba9-472d-9a77-5dac17d8469a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:34.468493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bde7b0-aba9-472d-9a77-5dac17d8469a None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:34.470055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 31/62] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:34.475624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704ef53-98c1-4476-bde0-e41f351f871d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:34.476528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704ef53-98c1-4476-bde0-e41f351f871d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:34.477024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704ef53-98c1-4476-bde0-e41f351f871d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:34.500145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c704ef53-98c1-4476-bde0-e41f351f871d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:34.502271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c704ef53-98c1-4476-bde0-e41f351f871d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:34.548019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34767c-28a4-4757-ac83-9c781090054e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:34.548599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34767c-28a4-4757-ac83-9c781090054e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:34.549058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34767c-28a4-4757-ac83-9c781090054e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:34.550559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 32/63] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:34.556385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aa16a-77ea-42db-9523-65a6739d3ccd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:34.556935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aa16a-77ea-42db-9523-65a6739d3ccd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:34.557374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aa16a-77ea-42db-9523-65a6739d3ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:34.584578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-384aa16a-77ea-42db-9523-65a6739d3ccd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:34.587758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-384aa16a-77ea-42db-9523-65a6739d3ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:34.621589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 32/64] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:34.636283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:34.706334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 33/65] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:34.772741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623b464-edd2-4416-b41d-acc57a567395 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:34.773044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623b464-edd2-4416-b41d-acc57a567395 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:34.773223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623b464-edd2-4416-b41d-acc57a567395 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:34.774599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 33/66] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:34.786781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b3bd0-35a7-4ed0-8448-d5f1bbadc390 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:34.787073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b3bd0-35a7-4ed0-8448-d5f1bbadc390 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:34.787347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b3bd0-35a7-4ed0-8448-d5f1bbadc390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:34.807794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d90b3bd0-35a7-4ed0-8448-d5f1bbadc390 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:34.809036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d90b3bd0-35a7-4ed0-8448-d5f1bbadc390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:34.906127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 34/67] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:34.933497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623b464-edd2-4416-b41d-acc57a567395 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:34.996965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b3bd0-35a7-4ed0-8448-d5f1bbadc390 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.070346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.110654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623b464-edd2-4416-b41d-acc57a567395 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.143196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.158164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9b86-fd8a-4a35-a9d9-74fb43fdc5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.158811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9b86-fd8a-4a35-a9d9-74fb43fdc5d1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.159097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9b86-fd8a-4a35-a9d9-74fb43fdc5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.159368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9b86-fd8a-4a35-a9d9-74fb43fdc5d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.159676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b3bd0-35a7-4ed0-8448-d5f1bbadc390 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.169915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa9b86-fd8a-4a35-a9d9-74fb43fdc5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-2efa9b86-fd8a-4a35-a9d9-74fb43fdc5d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:35.174763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:35.175609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:35.176417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 34/68] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:35.177492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7306-06d1-4a91-83c5-b4a446ee21a1 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.177771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7306-06d1-4a91-83c5-b4a446ee21a1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.178111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7306-06d1-4a91-83c5-b4a446ee21a1 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.179520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 35/69] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:35.184495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c246-c896-4d23-af48-b229aeed49a2 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.184780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c246-c896-4d23-af48-b229aeed49a2 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.185054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c246-c896-4d23-af48-b229aeed49a2 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.190142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 36/70] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:35.190142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c246-c896-4d23-af48-b229aeed49a2 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.205980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936db8b-5236-4224-9306-11788fd006c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.206798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936db8b-5236-4224-9306-11788fd006c9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.207143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936db8b-5236-4224-9306-11788fd006c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.207586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936db8b-5236-4224-9306-11788fd006c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.218205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936db8b-5236-4224-9306-11788fd006c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-0936db8b-5236-4224-9306-11788fd006c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:35.223027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:35.223920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:35.224743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 35/71] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:35.229607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad841b7-d48f-4518-9072-71b96c6d712e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.229889 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad841b7-d48f-4518-9072-71b96c6d712e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.230168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad841b7-d48f-4518-9072-71b96c6d712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.247623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-bad841b7-d48f-4518-9072-71b96c6d712e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:35.248850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bad841b7-d48f-4518-9072-71b96c6d712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:35.290346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c246-c896-4d23-af48-b229aeed49a2 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.312579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 36/72] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:35.313802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6bab-2ae7-4753-b77b-0748d693812c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.314069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6bab-2ae7-4753-b77b-0748d693812c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.314340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6bab-2ae7-4753-b77b-0748d693812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.324848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.325630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.326073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.326854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.327812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:35.328530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:35.330077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:35.331781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-a26f6bab-2ae7-4753-b77b-0748d693812c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:35.332978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a26f6bab-2ae7-4753-b77b-0748d693812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:35.340019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 37/73] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:35.341480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.350073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.401620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 37/74] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:35.408102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f6bab-2ae7-4753-b77b-0748d693812c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.411225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.471280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf83dd7-eacf-45f9-b680-6748c697482f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.506252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.506981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.507246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.507517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.518554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:35.522219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f6bab-2ae7-4753-b77b-0748d693812c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:35.524046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:35.524897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:35.525660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 38/75] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:35.530908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.568490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.569102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.569380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.569701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.570370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:35.575233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:35.577349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:35.587105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 38/76] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:35.588156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.598644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.639210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027525d-ce1b-42f9-ac18-125fc0c4234c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.656598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.671980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.672615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.672890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.673160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] PATH_INFO: `/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.674284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] RBAC: Authorizing `identity:get_user(user_id=6a7ff5148f6b41108f35cb9aa85b66e2)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:35.675751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:35.677359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 39/77] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:35.678112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.692376 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adae595-90fa-49c0-b329-bc0778b34ce8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.724658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.725269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.725546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.725845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.726509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:35.726810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:35.728214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:35.735905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 39/78] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:35.736680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.772841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12919e75-2309-4f39-887d-ff66d58d2d6e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.801166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.806462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.807095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.807376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.807655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.817840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:35.823140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:35.824117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:35.824918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 40/79] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:35.825893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.838531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311d4e-f2b5-4103-a56e-e18934ece4c6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.869040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:35.876162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.876791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.877072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.877345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.893657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:35.895324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:35.897002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 40/80] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:35.898778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:35.927498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9973c94-512d-43ec-8456-9adafa9e9f93 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:35.963103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:35.963719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:35.964044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:35.964311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:35.965431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:35.966883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:35.973145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 41/81] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:35.973145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.005561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251fc73-0032-4011-8503-1963ef449932 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.026894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.040635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.041253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.041535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.041809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.066248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f79f-7931-4c36-9aa3-c0ec26680d3a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.096159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.099726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.100394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.100662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.100996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.102118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.103553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.105141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 42/82] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:36.105889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.113274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.114685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.202931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4729d16-4293-4eb7-bea9-45667f35d9b7 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.209722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 41/83] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:36.210601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.232245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.240519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.241139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.241495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.241791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.242466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:36.242774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.244188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.252513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 43/84] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:36.253123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.301677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.313764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf780c5-b7a9-4f62-be79-264cfa2f78f9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.347156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.347702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.348021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.348300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.348546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76bf1-ab8f-45ce-9929-9929bbe618e6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.382333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.382944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.383223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.383493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.384186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.385629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.407562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 44/85] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:37:36 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:36.408253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.433329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.435152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.460780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.524354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db129279-b28a-40fb-9a0e-02cf79fd895c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.542110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.558598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 42/86] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:36.559478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.568140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.568778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.569059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.569331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.645505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.646933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.682748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8d83-a977-422a-85ff-8ab7f9679ae2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.701761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.725962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.726729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.727420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.727710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.729208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.730848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.733268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 43/87] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:36.733892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.766130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 45/88] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:36.767078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.811654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:36.861516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf558-a96f-460c-8b36-bc7956846752 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.874752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998a092-9b87-4d80-abaf-438dc0bfa7cc None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:36.905032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.905591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.905869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.906145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.907668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.909239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.911259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 44/89] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:36.912257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.922554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:36.923220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:36.923506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:36.923777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:36.924995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:36.926568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:36.928319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 46/90] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:36.929236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:36.956316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:37.020245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3146154-f2aa-4eb7-9899-6280a1403de9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:37.035160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccd565-c774-4e86-b352-dc2a02a27bd0 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:37.044672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:37.057772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:37.058427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:37.058710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:37.058982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:37.059708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=6a7ff5148f6b41108f35cb9aa85b66e2) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:37.060046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:37.061482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:37.067478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 45/91] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/role_assignments?user.id=6a7ff5148f6b41108f35cb9aa85b66e2&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:37.068138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:37.071786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:37.072466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:37.072748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:37.073023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:37.074196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:37.075687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:37.077443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 47/92] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:36 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:37.078250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:37.188815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c3340-fe30-49c8-a669-ddeaf932071a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:37.227297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:37.227954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:37.228297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:37.228574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:37.297798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:37.301508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:37.308664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:37.310253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:37.381886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccf2e-5b2b-43a0-b822-cc56c5f07e4f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:37.422058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 46/93] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:37.423193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:37.464062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:37.464959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:37.467030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:37.467488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:37.468451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=e2fd43b4750842c794560ebef09e5f94) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:37.470219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:37.472855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:37.485278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 48/94] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:37 2022] GET /identity/v3/role_assignments?user.id=e2fd43b4750842c794560ebef09e5f94&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 259 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:37.490190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:37.656294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:37.684077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcdf80-33d9-4d37-9d11-da6f4ad88403 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:37.706281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9e700-1307-4a93-8491-147cf04c1907 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:37.763105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:37.763652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:37.763969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:37.764250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:37.765062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] RBAC: Adding query filter params (role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=e2fd43b4750842c794560ebef09e5f94) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:37.765359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:37.767432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:37.774765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 47/95] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:37 2022] GET /identity/v3/role_assignments?user.id=e2fd43b4750842c794560ebef09e5f94&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 259 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:37.789466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512fe01-4b49-464f-b9a2-48794c3bac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:37.790497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512fe01-4b49-464f-b9a2-48794c3bac77 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:37.790978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512fe01-4b49-464f-b9a2-48794c3bac77 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:37.791452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512fe01-4b49-464f-b9a2-48794c3bac77 None admin] PATH_INFO: `/v3/projects/c3690411676d41fcbb4cc36a8d3435fa` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:37.796330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512fe01-4b49-464f-b9a2-48794c3bac77 None admin] RBAC: Authorizing `identity:get_project(project_id=c3690411676d41fcbb4cc36a8d3435fa)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:37.798209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512fe01-4b49-464f-b9a2-48794c3bac77 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:37.800308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 49/96] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:37 2022] GET /identity/v3/projects/c3690411676d41fcbb4cc36a8d3435fa => generated 312 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:37.806882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:37.927722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1457b77-6d33-49e9-8fb4-c53ce62868b4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:37.964430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf04d4-a21e-47c7-8f87-7dd373c3a670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:37.965046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf04d4-a21e-47c7-8f87-7dd373c3a670 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:37.965330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf04d4-a21e-47c7-8f87-7dd373c3a670 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:37.965605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf04d4-a21e-47c7-8f87-7dd373c3a670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:37.968535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf04d4-a21e-47c7-8f87-7dd373c3a670 None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=c3690411676d41fcbb4cc36a8d3435fa, user.id=e2fd43b4750842c794560ebef09e5f94) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:37.968535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf04d4-a21e-47c7-8f87-7dd373c3a670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:37.968535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf04d4-a21e-47c7-8f87-7dd373c3a670 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:37.974450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 48/97] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:37 2022] GET /identity/v3/role_assignments?user.id=e2fd43b4750842c794560ebef09e5f94&scope.project.id=c3690411676d41fcbb4cc36a8d3435fa&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:39.332551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19ae40-2879-40c7-be0e-494a47c940fc None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:39.333414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19ae40-2879-40c7-be0e-494a47c940fc None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:39.333900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19ae40-2879-40c7-be0e-494a47c940fc None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:39.335455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 50/98] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:39.349392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df059cd-f8be-41e9-84e2-54a0e508d98d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:39.349874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df059cd-f8be-41e9-84e2-54a0e508d98d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:39.350156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df059cd-f8be-41e9-84e2-54a0e508d98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:39.373293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4df059cd-f8be-41e9-84e2-54a0e508d98d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:39.374504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4df059cd-f8be-41e9-84e2-54a0e508d98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:39.386363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:39.447364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 49/99] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:39.486587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36f2a-870e-44ec-8472-9b31f39b0b7c None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:39.486886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36f2a-870e-44ec-8472-9b31f39b0b7c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:39.487144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36f2a-870e-44ec-8472-9b31f39b0b7c None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:39.488537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 51/100] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:39.504075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75714ae5-9a00-4d82-9919-7f53d159dd91 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:39.504703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75714ae5-9a00-4d82-9919-7f53d159dd91 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:39.505172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75714ae5-9a00-4d82-9919-7f53d159dd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:39.531067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-75714ae5-9a00-4d82-9919-7f53d159dd91 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:39.532640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-75714ae5-9a00-4d82-9919-7f53d159dd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:39.604633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 50/101] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:39.897038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36f2a-870e-44ec-8472-9b31f39b0b7c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:39.913877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:40.036984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36f2a-870e-44ec-8472-9b31f39b0b7c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:40.087435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:40.088446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.088941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.089409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.090744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:40.092421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:40.094192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 52/102] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:40.100133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75714ae5-9a00-4d82-9919-7f53d159dd91 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:40.158481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:40.270564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:40.309592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:40.324274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75714ae5-9a00-4d82-9919-7f53d159dd91 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:40.341964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59476287-9102-4fe8-ac46-bc6b3e9fde16 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:40.390891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:40.391565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.391798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.392105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.392773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:40.395544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:40.408823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f603eb-5081-4196-b96a-b0078218b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:40.409403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f603eb-5081-4196-b96a-b0078218b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.409681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f603eb-5081-4196-b96a-b0078218b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.409951 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f603eb-5081-4196-b96a-b0078218b1c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.426921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f603eb-5081-4196-b96a-b0078218b1c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-18f603eb-5081-4196-b96a-b0078218b1c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:40.435558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:40.436453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:40.437224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:40.438747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 53/103] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:40.441085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 51/104] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:37:40 2022] POST /identity/v3/projects => generated 302 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:40.449200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:40.450189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2433e79-015e-4872-a604-37143a7983f5 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.450480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2433e79-015e-4872-a604-37143a7983f5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.450755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2433e79-015e-4872-a604-37143a7983f5 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.456269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 54/105] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:40 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:40.466590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12f20-bcbc-4817-ba42-38796258f1b3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.466897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12f20-bcbc-4817-ba42-38796258f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.467139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12f20-bcbc-4817-ba42-38796258f1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.517195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-bcb12f20-bcbc-4817-ba42-38796258f1b3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:40.518504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-bcb12f20-bcbc-4817-ba42-38796258f1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:40.665764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44d51c-6ad7-471f-a948-2acc45517911 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:40.710101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:40.721800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09281bd4-2684-45e5-9453-514b1ca32151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:40.722423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09281bd4-2684-45e5-9453-514b1ca32151 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.722705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09281bd4-2684-45e5-9453-514b1ca32151 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.722982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09281bd4-2684-45e5-9453-514b1ca32151 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.723662 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09281bd4-2684-45e5-9453-514b1ca32151 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:40.724025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09281bd4-2684-45e5-9453-514b1ca32151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:40.725398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09281bd4-2684-45e5-9453-514b1ca32151 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:40.731343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 55/106] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:40.732925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ca0b6-b430-430f-83c1-aba304978bc3 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.733214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ca0b6-b430-430f-83c1-aba304978bc3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.733491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ca0b6-b430-430f-83c1-aba304978bc3 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.734882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 56/107] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:40.735294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 52/108] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:40.735738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cd8a-6a75-403f-a3d4-365d5421550b None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.736046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cd8a-6a75-403f-a3d4-365d5421550b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.736320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cd8a-6a75-403f-a3d4-365d5421550b None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.737654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 57/109] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:40.741299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb92d9-9eed-4f4c-b14e-12887ffced2a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.741883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb92d9-9eed-4f4c-b14e-12887ffced2a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.742354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb92d9-9eed-4f4c-b14e-12887ffced2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.745795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953cd8a-6a75-403f-a3d4-365d5421550b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:40.768341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e0bb92d9-9eed-4f4c-b14e-12887ffced2a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:40.769544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e0bb92d9-9eed-4f4c-b14e-12887ffced2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:40.848908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:40.868138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 53/110] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:40.869878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac6b9-e35c-4972-beea-2c6569087504 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.870362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac6b9-e35c-4972-beea-2c6569087504 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.870805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac6b9-e35c-4972-beea-2c6569087504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.888717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953cd8a-6a75-403f-a3d4-365d5421550b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:40.891178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-00cac6b9-e35c-4972-beea-2c6569087504 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:40.892377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-00cac6b9-e35c-4972-beea-2c6569087504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:40.943541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4face78a-2c35-477a-9166-3177d2ff611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:40.944528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4face78a-2c35-477a-9166-3177d2ff611a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:40.945001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4face78a-2c35-477a-9166-3177d2ff611a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:40.945527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4face78a-2c35-477a-9166-3177d2ff611a None admin] PATH_INFO: `/v3/users/e1d77db0bf8f4261a196a1a47603291a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:40.963039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 54/111] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:40.979922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4face78a-2c35-477a-9166-3177d2ff611a None admin] RBAC: Authorizing `identity:get_user(user_id=e1d77db0bf8f4261a196a1a47603291a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:40.982210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4face78a-2c35-477a-9166-3177d2ff611a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:40.985031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 58/112] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:40 2022] GET /identity/v3/users/e1d77db0bf8f4261a196a1a47603291a => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:40.990326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cac6b9-e35c-4972-beea-2c6569087504 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.060397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:41.126736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cac6b9-e35c-4972-beea-2c6569087504 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:41.181694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:41.182320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.182604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.182882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.222364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd29d31-c317-4748-8147-761cd6ea65d1 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.222648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd29d31-c317-4748-8147-761cd6ea65d1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.222924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd29d31-c317-4748-8147-761cd6ea65d1 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.224445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 59/113] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:41.237227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6211b-5631-48ab-a862-acac69e1cd4f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.237519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6211b-5631-48ab-a862-acac69e1cd4f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.237797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6211b-5631-48ab-a862-acac69e1cd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.258761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-cea6211b-5631-48ab-a862-acac69e1cd4f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:41.259975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-cea6211b-5631-48ab-a862-acac69e1cd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:41.272707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:41.274383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:41.326304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:41.360361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 60/114] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:41.361854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6211b-5631-48ab-a862-acac69e1cd4f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.400907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 55/115] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:41.402382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.441461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:41.499790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6211b-5631-48ab-a862-acac69e1cd4f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:41.515208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e24f0-c11c-40f1-a741-40a8ad7442b3 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:41.550199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:41.551274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.551707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.552015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.559730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:41.560921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.561769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.562290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.564214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:41.569128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:41.570048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 61/116] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:41.571508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.579213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:41.586387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:41.587250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:41.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:41.594421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 56/117] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:41.594421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.608216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:41.689724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfdaa8-132e-4a42-ac2f-dc198e36bf6d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:41.714130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088e5a5-7198-4f8c-9f4f-901f7c9b9298 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:41.714333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:41.732573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:41.733225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.733505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.733803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.734965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:41.736495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:41.743625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 62/118] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:41.743625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.758373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:41.758939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.759216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.759484 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.770237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:41.774837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:41.775669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 57/119] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:41.777174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.846199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaee6b-2796-470c-8a7a-e1a8d8868951 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:41.846974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:41.880245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:41.880522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dd8db-953e-435f-beec-a6f0f80939b7 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:41.880865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.881144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.881415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.882138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:41.882444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:41.883822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:41.891931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 63/120] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:41.896808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.919442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:41.920240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.920844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.921125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.921937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:41.922242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:41.923614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:41.936823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 58/121] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:41.936823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:41.950638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.011571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8566d-18fc-4339-98b5-e337aa241529 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.041971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac31be-34cd-4a4d-9cb1-1a2c05c6c3cd None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.047576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.048474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.048941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.049222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.060289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:42.065635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:42.066562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:42.067335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 64/122] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:42.067996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.077691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.084242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.084857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.085163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.085458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.086186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] RBAC: Adding query filter params (role.id=3fff797d04c14612b73559493d44d1fd, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=e1d77db0bf8f4261a196a1a47603291a) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:42.086514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.087975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.093754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 59/123] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:41 2022] GET /identity/v3/role_assignments?user.id=e1d77db0bf8f4261a196a1a47603291a&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=3fff797d04c14612b73559493d44d1fd => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.094452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.223227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ef94f-0dda-4ef4-979b-b9cbf54753c4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.235606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.240065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3258ea0-54a8-4e0e-90fc-ab9557b33ac2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.269538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.270731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.271621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.272413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.273483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:42.273800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.275541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.285538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 65/124] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.286171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.286550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.287397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.287873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.288435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] PATH_INFO: `/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.289819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] RBAC: Authorizing `identity:get_user(user_id=6a7ff5148f6b41108f35cb9aa85b66e2)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.291468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.293682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 60/125] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.294724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.358601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.405033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35881c-4922-4f5d-8160-22b3629ef244 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.419099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5241c-ab73-4ae0-bf0c-92f40ee31140 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.441967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.442951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.443450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.443970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.444913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:42.445400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.446965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.456028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 61/126] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.457130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.466938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.467589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.467870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.468186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.469341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.471466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.473988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 66/127] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.475039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.532741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.580501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcc49-e82d-4350-a01f-afd440bf3251 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.606750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.615628 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0aa558-699f-4dd0-908c-f75e71c0a3fe None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.617708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.618560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.619051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.619508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.620865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.622812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.624719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 62/128] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.625719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.668642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.669306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.669591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.669923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.671168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.673314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.675782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 67/129] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.677081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.735379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb7d77-4f0b-4efa-885b-405675460a22 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.771308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.772316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.772816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.773275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.774695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.776437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.778263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 63/130] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.779218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.809375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d52a58-c4a4-4f99-a7c6-6bb19e40844f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.818995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.846655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.847313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.847859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.847985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.848248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.849869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.852098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.853786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 68/131] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.854524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.894053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca4945-22bb-4055-9c49-5fb0f2b899be None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.930175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.931223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.931732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.932206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.933537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.935227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.937438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 64/132] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:42.938424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.977577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb34788-e5b5-4aa8-be65-109acf2c785f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:43.015244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:43.015954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:43.016249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:43.016524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/6a7ff5148f6b41108f35cb9aa85b66e2/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:43.018467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec27f0f03e8249ff8c379f6356fc3491, user_id=6a7ff5148f6b41108f35cb9aa85b66e2, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:43.019948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:43.032649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 69/133] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:42 2022] PUT /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/6a7ff5148f6b41108f35cb9aa85b66e2/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:37:43.033225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:43.062852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cec41c-114c-42ce-9340-196c806b4aaa None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:43.077097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:43.110692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:43.120700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:43.121916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:43.122332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:43.122739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] PATH_INFO: `/v3/projects/c3690411676d41fcbb4cc36a8d3435fa` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:43.124137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] RBAC: Authorizing `identity:get_project(project_id=c3690411676d41fcbb4cc36a8d3435fa)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:43.126509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:43.129079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 65/134] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/projects/c3690411676d41fcbb4cc36a8d3435fa => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:43.130066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:43.159568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4e7cf-cedc-4578-bc61-5b867b2cabd2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:43.197977 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad347d-0142-43fe-8a37-d6bc2e08a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:43.198821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad347d-0142-43fe-8a37-d6bc2e08a6e4 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:43.199199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad347d-0142-43fe-8a37-d6bc2e08a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:43.199567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad347d-0142-43fe-8a37-d6bc2e08a6e4 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/e2fd43b4750842c794560ebef09e5f94/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:43.201759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad347d-0142-43fe-8a37-d6bc2e08a6e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec27f0f03e8249ff8c379f6356fc3491, user_id=e2fd43b4750842c794560ebef09e5f94, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:43.203442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad347d-0142-43fe-8a37-d6bc2e08a6e4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:43.223460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 70/135] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:43 2022] PUT /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/e2fd43b4750842c794560ebef09e5f94/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:37:43.225406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2061b8a-ed03-4886-859c-3b2ce13e1dd9 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:43.225809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2061b8a-ed03-4886-859c-3b2ce13e1dd9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:43.226968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2061b8a-ed03-4886-859c-3b2ce13e1dd9 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:43.228479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 71/136] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:43.229325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2061b8a-ed03-4886-859c-3b2ce13e1dd9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:43.262724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f612-9218-45c5-b8db-fd58e728012b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:43.308083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8426c-2f47-4e61-8214-0cab21f08b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:43.309089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8426c-2f47-4e61-8214-0cab21f08b0c None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:43.309991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8426c-2f47-4e61-8214-0cab21f08b0c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:43.310384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8426c-2f47-4e61-8214-0cab21f08b0c None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/e2fd43b4750842c794560ebef09e5f94/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:43.313211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8426c-2f47-4e61-8214-0cab21f08b0c None admin] RBAC: Authorizing `identity:create_grant(project_id=ec27f0f03e8249ff8c379f6356fc3491, user_id=e2fd43b4750842c794560ebef09e5f94, role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:43.314777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8426c-2f47-4e61-8214-0cab21f08b0c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:43.341970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 66/137] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:43 2022] PUT /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/e2fd43b4750842c794560ebef09e5f94/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:37:43.343443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894dff3-9ffc-4045-9744-b3e49d4d471b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:43.343819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894dff3-9ffc-4045-9744-b3e49d4d471b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:43.344217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894dff3-9ffc-4045-9744-b3e49d4d471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:43.392335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:43.397173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2894dff3-9ffc-4045-9744-b3e49d4d471b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:43.398903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2894dff3-9ffc-4045-9744-b3e49d4d471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:43.410897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2061b8a-ed03-4886-859c-3b2ce13e1dd9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:43.482870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:43.483759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:43.484132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:43.484504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] PATH_INFO: `/v3/projects/c3690411676d41fcbb4cc36a8d3435fa/users/e2fd43b4750842c794560ebef09e5f94/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:43.489362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3690411676d41fcbb4cc36a8d3435fa, user_id=e2fd43b4750842c794560ebef09e5f94, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:43.490957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:43.504335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:43.516090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 72/138] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:43 2022] PUT /identity/v3/projects/c3690411676d41fcbb4cc36a8d3435fa/users/e2fd43b4750842c794560ebef09e5f94/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:37:43.521667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 67/139] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:43.881630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:44.017482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8db134-88c5-44bc-b8f9-84d82cd7e7e4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:44.065407 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d1648-3782-402e-8dd8-d604c00071ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:44.066057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d1648-3782-402e-8dd8-d604c00071ea None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:44.066348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d1648-3782-402e-8dd8-d604c00071ea None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:44.066619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d1648-3782-402e-8dd8-d604c00071ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:44.067759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d1648-3782-402e-8dd8-d604c00071ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:44.069313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d1648-3782-402e-8dd8-d604c00071ea None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:44.070916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 73/140] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:44.076635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894dff3-9ffc-4045-9744-b3e49d4d471b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:44.209292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894dff3-9ffc-4045-9744-b3e49d4d471b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:44.253428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56863f67-e67d-4e1f-b125-fd9d9806d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:44.254053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863f67-e67d-4e1f-b125-fd9d9806d95e None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:44.254337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863f67-e67d-4e1f-b125-fd9d9806d95e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:44.254616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863f67-e67d-4e1f-b125-fd9d9806d95e None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:44.255288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56863f67-e67d-4e1f-b125-fd9d9806d95e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:44.256826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56863f67-e67d-4e1f-b125-fd9d9806d95e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:44.288933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 68/141] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:37:44 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:44.692562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479dc10-6c6a-4f93-9824-112810d8f559 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:44.693169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479dc10-6c6a-4f93-9824-112810d8f559 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:44.693169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479dc10-6c6a-4f93-9824-112810d8f559 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:44.694481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 74/142] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:44.711374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b7593-7d72-47d3-a56b-d5fda9e6ff8f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:44.711875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b7593-7d72-47d3-a56b-d5fda9e6ff8f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:44.712003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b7593-7d72-47d3-a56b-d5fda9e6ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:44.731451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-718b7593-7d72-47d3-a56b-d5fda9e6ff8f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:44.732671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-718b7593-7d72-47d3-a56b-d5fda9e6ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:44.777677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:44.813211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 69/143] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:45.345082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479dc10-6c6a-4f93-9824-112810d8f559 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:45.379594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:45.482630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479dc10-6c6a-4f93-9824-112810d8f559 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:45.526868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:45.527511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.527792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.528084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.543539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:45.548386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:45.549291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:45.550237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:45.551534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 75/144] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:45.557976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b7593-7d72-47d3-a56b-d5fda9e6ff8f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:45.695148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b7593-7d72-47d3-a56b-d5fda9e6ff8f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:45.734796 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f50d7-10cc-48bf-86c3-b5b1689fa1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:45.735575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f50d7-10cc-48bf-86c3-b5b1689fa1c9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.735736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f50d7-10cc-48bf-86c3-b5b1689fa1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.735965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f50d7-10cc-48bf-86c3-b5b1689fa1c9 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.736654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f50d7-10cc-48bf-86c3-b5b1689fa1c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:45.736952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f50d7-10cc-48bf-86c3-b5b1689fa1c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:45.738414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f50d7-10cc-48bf-86c3-b5b1689fa1c9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:45.748392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 70/145] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:45.755952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:45.868394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8208-00a2-44a0-bd69-17d4946b4f53 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.868727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8208-00a2-44a0-bd69-17d4946b4f53 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.868938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8208-00a2-44a0-bd69-17d4946b4f53 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.870296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 71/146] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:45.876904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec1e8-d206-472b-9eda-bb6ee691706b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.877178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec1e8-d206-472b-9eda-bb6ee691706b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.877442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec1e8-d206-472b-9eda-bb6ee691706b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.895416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:45.907990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-391ec1e8-d206-472b-9eda-bb6ee691706b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:45.909243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-391ec1e8-d206-472b-9eda-bb6ee691706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:45.924778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d309ce7-2d81-449a-9884-d8a03f887afc None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:45.977887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:45.990886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:45.991596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.991876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.992187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] PATH_INFO: `/v3/users/e1d77db0bf8f4261a196a1a47603291a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.997114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] RBAC: Authorizing `identity:get_user(user_id=e1d77db0bf8f4261a196a1a47603291a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:45.998577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:46.001234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 76/147] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:45 2022] GET /identity/v3/users/e1d77db0bf8f4261a196a1a47603291a => generated 289 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:46.006852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:46.090426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 72/148] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:46.162107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e9d7b-7584-4e15-9566-6f2fc1d2ac95 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:46.207917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:46.208565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:46.208844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:46.209117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:46.212554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:46.214021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:46.215918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 77/149] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:46 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:46.221513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec1e8-d206-472b-9eda-bb6ee691706b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:46.353448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec1e8-d206-472b-9eda-bb6ee691706b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:46.413608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54a457-4711-416f-aabc-2d0c90ec10a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:46.414295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54a457-4711-416f-aabc-2d0c90ec10a5 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:46.414572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54a457-4711-416f-aabc-2d0c90ec10a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:46.414838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54a457-4711-416f-aabc-2d0c90ec10a5 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/e1d77db0bf8f4261a196a1a47603291a/roles/3fff797d04c14612b73559493d44d1fd` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:46.421165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54a457-4711-416f-aabc-2d0c90ec10a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec27f0f03e8249ff8c379f6356fc3491, user_id=e1d77db0bf8f4261a196a1a47603291a, role_id=3fff797d04c14612b73559493d44d1fd)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:46.422642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54a457-4711-416f-aabc-2d0c90ec10a5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:46.440091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 73/150] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:46 2022] PUT /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/users/e1d77db0bf8f4261a196a1a47603291a/roles/3fff797d04c14612b73559493d44d1fd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:37:46.663011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:46.693720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:46.753873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f712-e95e-4258-9f85-0a8cfeab184c None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:46.754842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f712-e95e-4258-9f85-0a8cfeab184c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:46.755317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526f712-e95e-4258-9f85-0a8cfeab184c None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:46.758372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 74/151] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:46.763566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de1cf-0fd5-41ed-a76c-567eec7aad97 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:46.764158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de1cf-0fd5-41ed-a76c-567eec7aad97 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:46.764641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de1cf-0fd5-41ed-a76c-567eec7aad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:46.801046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-792de1cf-0fd5-41ed-a76c-567eec7aad97 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:46.802708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-792de1cf-0fd5-41ed-a76c-567eec7aad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:46.812747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353d1ba-57f2-4b31-bd77-ac43c41c9502 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:46.851520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:46.871338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:46.872023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:46.872305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:46.872578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:46.899247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:46.904751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:46.905837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:46.906816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:46.909698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 78/152] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:46.915598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:46.987201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 75/153] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:46.988575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856055c4-c675-4666-a29b-ce76629449bc None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:46.988820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856055c4-c675-4666-a29b-ce76629449bc None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:46.989093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856055c4-c675-4666-a29b-ce76629449bc None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:46.990372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 76/154] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:46.998571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1c2d2-65a8-44f2-8526-cc068b71071a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:46.998855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1c2d2-65a8-44f2-8526-cc068b71071a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:46.999117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1c2d2-65a8-44f2-8526-cc068b71071a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:47.034809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-43b1c2d2-65a8-44f2-8526-cc068b71071a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:47.036027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-43b1c2d2-65a8-44f2-8526-cc068b71071a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:47.089125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:47.122666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fc89b-7cbe-49bb-a7c5-07b40e1d298f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:47.143503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 77/155] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:47.182818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71cc21-2661-4f06-ae23-210a654fbf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:47.183352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71cc21-2661-4f06-ae23-210a654fbf76 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:47.183624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71cc21-2661-4f06-ae23-210a654fbf76 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:47.183908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71cc21-2661-4f06-ae23-210a654fbf76 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:47.184574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71cc21-2661-4f06-ae23-210a654fbf76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:47.184928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71cc21-2661-4f06-ae23-210a654fbf76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:47.186367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71cc21-2661-4f06-ae23-210a654fbf76 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:47.198614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 79/156] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:47.204414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1c2d2-65a8-44f2-8526-cc068b71071a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:47.253557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5d557-6b98-48f7-bbff-9a9eb52cf212 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:47.253862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5d557-6b98-48f7-bbff-9a9eb52cf212 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:47.254108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5d557-6b98-48f7-bbff-9a9eb52cf212 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:47.255465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 80/157] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:47.261066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685539e-83d8-4bed-a1e3-b9e74f8ae7af None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:47.261345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685539e-83d8-4bed-a1e3-b9e74f8ae7af None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:47.261611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685539e-83d8-4bed-a1e3-b9e74f8ae7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:47.296969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5685539e-83d8-4bed-a1e3-b9e74f8ae7af None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:47.298155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5685539e-83d8-4bed-a1e3-b9e74f8ae7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:47.322095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:47.340490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1c2d2-65a8-44f2-8526-cc068b71071a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:47.383993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:47.384642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:47.384923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:47.385483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] PATH_INFO: `/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:47.386623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] RBAC: Authorizing `identity:get_user(user_id=6a7ff5148f6b41108f35cb9aa85b66e2)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:47.388105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:47.389695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 78/158] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:47 2022] GET /identity/v3/users/6a7ff5148f6b41108f35cb9aa85b66e2 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:47.396464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:47.425095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 81/159] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:47.530017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a96b-00a9-4af0-bb03-c521f717fc38 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:47.563715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685539e-83d8-4bed-a1e3-b9e74f8ae7af None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:47.581032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:47.581917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:47.582346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:47.582762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:47.643042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:47.730737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:47.732227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:47.732331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:47.793116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685539e-83d8-4bed-a1e3-b9e74f8ae7af None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:47.842786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:47.843427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:47.843696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:47.844003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:47.854321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:47.859280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:47.860186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:47.861005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 82/160] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:37:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:47.861645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:48.032524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 79/161] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:48.033401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:48.143187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f754-8412-4754-84af-1fff19e3c3e3 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:48.220272 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:48.224235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:48.240051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:48.241238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:48.241949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:48.242674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:48.268781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:48.274273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:48.275160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:48.276021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:48.283629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 83/162] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:48.285745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:48.421355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4090a-6f38-4a9c-a66e-af8011f3e063 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:48.504613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:48.505612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:48.506097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:48.506608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:48.507482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:48.508007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:48.509603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:48.530195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 80/163] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:37:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 497 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:48.531392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:48.653142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b57ba-1da7-4fd0-96ec-f92a1ef46a45 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:48.703049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c1956-f1a7-4ebd-a1c8-00997ad7668a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:48.725292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:48.751698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:48.752393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:48.752766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:48.753043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:48.755288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:48.756071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:48.756440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:48.756869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:48.762018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:48.762871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:48.764753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 81/164] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:48 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:48.765166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:48.785600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:48.797019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:48.797924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:48.798724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 84/165] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 514 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:48.801148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:48.909320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.018022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4072d-b66d-4cfd-bd3f-ae1235452fe1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.032320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c81b8-8e28-44a2-8356-1c07816299fd None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.074398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.075035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.075327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.075544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.076239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:49.076543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.077898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.090918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.091556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.091823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.092119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.093374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.094828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.096664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 85/166] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:48 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:49.097196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.099579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 82/167] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:49.100473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.199566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.253368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028a277-2fba-4e8d-a4ac-983b2ea7d2f2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.295665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ca0e3-a121-4ca8-ae4f-d236c5f22111 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.323735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce769f-9774-4f43-9040-783719247c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.325557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce769f-9774-4f43-9040-783719247c05 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.326211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce769f-9774-4f43-9040-783719247c05 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.327795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce769f-9774-4f43-9040-783719247c05 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.329655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce769f-9774-4f43-9040-783719247c05 None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:49.330705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce769f-9774-4f43-9040-783719247c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.334932 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce769f-9774-4f43-9040-783719247c05 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.336811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.353011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 83/168] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:49.355219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50101113-0190-4c3c-a558-845a8d30d18d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.355878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50101113-0190-4c3c-a558-845a8d30d18d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.356943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50101113-0190-4c3c-a558-845a8d30d18d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.359102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 84/169] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:49.361095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50101113-0190-4c3c-a558-845a8d30d18d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.366220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.367139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.367622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.368109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.369056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=6a7ff5148f6b41108f35cb9aa85b66e2) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:49.369729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.372556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.386720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 86/170] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/role_assignments?user.id=6a7ff5148f6b41108f35cb9aa85b66e2&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:49.389332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.509569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebabe-2667-476f-b1a2-931370bc9f5a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.549330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140d71d-6908-4f54-82bd-293837e4f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.550427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d71d-6908-4f54-82bd-293837e4f9db None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.550936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d71d-6908-4f54-82bd-293837e4f9db None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.551385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d71d-6908-4f54-82bd-293837e4f9db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.565333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50101113-0190-4c3c-a558-845a8d30d18d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.575461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.618748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.637003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.637565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140d71d-6908-4f54-82bd-293837e4f9db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.637933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.638336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.638748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.640094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.640464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140d71d-6908-4f54-82bd-293837e4f9db None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.641555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.643568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 85/171] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:49.644558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.827979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 87/172] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:49.831094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eba04-9a5b-44ad-b629-64035d53777f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.832408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eba04-9a5b-44ad-b629-64035d53777f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.832707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eba04-9a5b-44ad-b629-64035d53777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.843007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e6c37-a9d9-4f4b-9a0e-2fd06e5462ab None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.853485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-7f3eba04-9a5b-44ad-b629-64035d53777f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:49.855007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7f3eba04-9a5b-44ad-b629-64035d53777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:49.884089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.885120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.885651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.886163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] PATH_INFO: `/v3/users/e2fd43b4750842c794560ebef09e5f94` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.887775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] RBAC: Authorizing `identity:get_user(user_id=e2fd43b4750842c794560ebef09e5f94)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.889505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.891728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 86/173] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/users/e2fd43b4750842c794560ebef09e5f94 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:49.893109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.928877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 88/174] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:49.930145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b94da-045b-45d5-ac67-a6468e9ef3aa None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.930428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b94da-045b-45d5-ac67-a6468e9ef3aa None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.930698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b94da-045b-45d5-ac67-a6468e9ef3aa None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.936293 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 89/175] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:49.937154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b94da-045b-45d5-ac67-a6468e9ef3aa None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.955161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.957212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.026905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a477e7-ecfd-46b0-9e2f-9e841c500548 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.063018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b94da-045b-45d5-ac67-a6468e9ef3aa None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.064690 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b78e7-ef42-4554-a096-7480c3b387f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.066093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b78e7-ef42-4554-a096-7480c3b387f0 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.066391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b78e7-ef42-4554-a096-7480c3b387f0 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.066663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b78e7-ef42-4554-a096-7480c3b387f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.100685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.101302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.101581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.101851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.103018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:50.104522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:50.106203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 90/176] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:50.107040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:50.141666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b78e7-ef42-4554-a096-7480c3b387f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:50.143146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b78e7-ef42-4554-a096-7480c3b387f0 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:50.188193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.190940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.207733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6af593-e875-4c29-8f81-338179afb295 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.241070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.241677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.241952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.242216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.247708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 87/177] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:50.249063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466b30e-74d8-4f9b-b03a-b5ce14f27a98 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.249352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466b30e-74d8-4f9b-b03a-b5ce14f27a98 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.249628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466b30e-74d8-4f9b-b03a-b5ce14f27a98 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.251097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 88/178] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:50.253527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255399d-4ed1-4ebe-aaae-94b5afeb6b38 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.253527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255399d-4ed1-4ebe-aaae-94b5afeb6b38 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.253527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255399d-4ed1-4ebe-aaae-94b5afeb6b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.270870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f255399d-4ed1-4ebe-aaae-94b5afeb6b38 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:50.272133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f255399d-4ed1-4ebe-aaae-94b5afeb6b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:50.313183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:50.314648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:50.340631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 89/179] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:50.345259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65628fa7-f727-4eb6-a525-9e2a185f9714 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.345761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65628fa7-f727-4eb6-a525-9e2a185f9714 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.346022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65628fa7-f727-4eb6-a525-9e2a185f9714 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.348895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 90/180] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:50.348895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65628fa7-f727-4eb6-a525-9e2a185f9714 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:50.400277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.418965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 91/181] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:50.418965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:50.440154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.468846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65628fa7-f727-4eb6-a525-9e2a185f9714 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.504666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.505372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.505646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.506044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.506784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] RBAC: Adding query filter params (role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=e2fd43b4750842c794560ebef09e5f94) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:50.507086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:50.508666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:50.515012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 91/182] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/role_assignments?user.id=e2fd43b4750842c794560ebef09e5f94&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:50.515806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:50.536198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99610a4-f777-489e-9be1-ced0fbc6a7be None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.578687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afde53-0807-4141-8254-fcb2791af80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.579748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afde53-0807-4141-8254-fcb2791af80d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.580265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afde53-0807-4141-8254-fcb2791af80d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.580745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afde53-0807-4141-8254-fcb2791af80d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.592926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afde53-0807-4141-8254-fcb2791af80d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-23afde53-0807-4141-8254-fcb2791af80d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:50.598394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:50.599355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:50.600235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:50.602635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 92/183] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:50.604139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa6e07-22c0-4203-a7bc-43a0c2145815 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.604635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa6e07-22c0-4203-a7bc-43a0c2145815 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.605092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa6e07-22c0-4203-a7bc-43a0c2145815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.624240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-01aa6e07-22c0-4203-a7bc-43a0c2145815 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:50.625805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-01aa6e07-22c0-4203-a7bc-43a0c2145815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:50.647330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5b116-a9c5-4abb-81f4-9232566e2af6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.667444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.689115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bd7f-8f07-44b7-ade2-ad633451ca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.689923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bd7f-8f07-44b7-ade2-ad633451ca83 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.690256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bd7f-8f07-44b7-ade2-ad633451ca83 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.690531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bd7f-8f07-44b7-ade2-ad633451ca83 None admin] PATH_INFO: `/v3/projects/c3690411676d41fcbb4cc36a8d3435fa` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.691733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753bd7f-8f07-44b7-ade2-ad633451ca83 None admin] RBAC: Authorizing `identity:get_project(project_id=c3690411676d41fcbb4cc36a8d3435fa)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:50.693273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753bd7f-8f07-44b7-ade2-ad633451ca83 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:50.694962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 92/184] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/projects/c3690411676d41fcbb4cc36a8d3435fa => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:50.700671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e03a80-6056-4dd3-93cd-3f6049105251 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.700671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e03a80-6056-4dd3-93cd-3f6049105251 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.700671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e03a80-6056-4dd3-93cd-3f6049105251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.703353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 93/185] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:50.704666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa6e07-22c0-4203-a7bc-43a0c2145815 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:50.715023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.723743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-04e03a80-6056-4dd3-93cd-3f6049105251 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:50.724695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-04e03a80-6056-4dd3-93cd-3f6049105251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:50.816391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 93/186] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:50.824085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e03a80-6056-4dd3-93cd-3f6049105251 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:50.830798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa6e07-22c0-4203-a7bc-43a0c2145815 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.877144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.878251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.878668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.879301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.881221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:50.882925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:50.885526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 94/187] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:50.886762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:50.919575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e03a80-6056-4dd3-93cd-3f6049105251 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:50.938517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.961046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:50.962084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:50.962811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:50.963493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:50.964382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:50.964850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:50.966517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:50.969757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:50.980455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 94/188] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:50.981931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.001766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc028f-1209-4097-9bfd-3abb4e1e8b51 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.035523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.036178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.036455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.036722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.037903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.039354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.045608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 95/189] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:51.045608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.091761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea384df-4b60-45ea-8694-a59105f45414 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.126085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.126724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.126989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.127224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.127999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=c3690411676d41fcbb4cc36a8d3435fa, user.id=e2fd43b4750842c794560ebef09e5f94) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:51.128304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.129717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.135628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 95/190] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:50 2022] GET /identity/v3/role_assignments?user.id=e2fd43b4750842c794560ebef09e5f94&scope.project.id=c3690411676d41fcbb4cc36a8d3435fa&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:51.136416 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.147373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c4d72-cc0e-4a02-a03f-dea7d65ec112 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.157440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:51.185198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.185731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.186037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.186271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.187041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=e2fd43b4750842c794560ebef09e5f94) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:51.187348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.188853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.195971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 96/191] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/role_assignments?user.id=e2fd43b4750842c794560ebef09e5f94&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:51.201514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.232343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:51.295461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd4b33-77a0-4b71-8ad1-a042e9882fe8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.322117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6ea84-e90a-4ad0-a7b1-89584b91ece3 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.345363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.346640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.347065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.347480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.364860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:51.367222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.369498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.369498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.369498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] PATH_INFO: `/v3/users/8731abd36fdb4e5ca5cfe14ee3093575` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:51.370417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:51.371649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:51.372633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:51.373387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 96/192] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:51.373873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.391416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] RBAC: Authorizing `identity:get_user(user_id=8731abd36fdb4e5ca5cfe14ee3093575)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.393155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.395151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 97/193] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/users/8731abd36fdb4e5ca5cfe14ee3093575 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:51.395993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.435489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:51.498542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4691b6-4d08-4ae6-946d-710470bdcc71 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.510225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78829924-e276-4dcb-a646-36cdbade3139 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.519921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:51.532244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.532853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.533131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.533406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.534052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.535499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.543735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.544411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.544723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.544990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.557184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 97/194] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:37:51 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:51.557580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:51.557997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:51.562817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:51.563786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 98/195] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:51.569024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.669965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:51.676164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad34fdc-5eab-43c3-811a-469c333e0448 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.681334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a702f8-1406-476d-9416-8faec3c6be28 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.713397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.713930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.714213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.714483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.715164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:51.715465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.716897 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.721902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.722872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.723322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.723761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.726018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 98/196] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:51.726748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.779338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:51.809720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.811492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.865463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a3757-ece7-47e3-9568-32a2c5c63cf5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:51.908771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:51.909112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:51.909305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:51.909572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:51.910236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:51.910534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:51.911915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:51.921181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 99/197] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:51.922003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:51.931160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:51.944344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 99/198] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:51.945342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.036027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2060a2e-70f6-4286-901b-ddcea303f572 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.037691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.068535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebcdb7-345a-4fb9-b49a-7c63b5d28b8e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.074911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.075629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.076005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.076352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] PATH_INFO: `/v3/users/8731abd36fdb4e5ca5cfe14ee3093575` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.077545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] RBAC: Authorizing `identity:get_user(user_id=8731abd36fdb4e5ca5cfe14ee3093575)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.079065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.081060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 100/199] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/users/8731abd36fdb4e5ca5cfe14ee3093575 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:52.085825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.117636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.118826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.119269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.119701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] PATH_INFO: `/v3/users/e1d77db0bf8f4261a196a1a47603291a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.122331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] RBAC: Authorizing `identity:get_user(user_id=e1d77db0bf8f4261a196a1a47603291a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.124027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.125909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 100/200] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:51 2022] GET /identity/v3/users/e1d77db0bf8f4261a196a1a47603291a => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:52.126931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.192705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.207576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d096eef-5642-4c8d-b9de-e587e951da08 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.236203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282c1df-83b2-4850-a636-ccfdfa552a6d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.246867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.248172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.248615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.249049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.250586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.252941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.255299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 101/201] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:52.256257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.274130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.274750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.275028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.275297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.307101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.359286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.361094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.379027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41605201-0ff7-452e-83f5-37b06a108762 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.418490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.419419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.420088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.420963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.434521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.488992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 101/202] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:52.493855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.505571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.507965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.556162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.605508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073516b2-56f2-4939-8ae7-4cfcfc361979 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.628547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 102/203] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:52.630083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.641615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.642227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.642502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.642767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.643495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=8731abd36fdb4e5ca5cfe14ee3093575) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:52.643797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.645274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.651559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 102/204] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/role_assignments?user.id=8731abd36fdb4e5ca5cfe14ee3093575&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:52.652368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.703059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.759141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d009ec-97c8-4d14-88be-935a0a6cf1a9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.800171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.801175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.801660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.802115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.803573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.805474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.807697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 103/205] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:52.814209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.833281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.848485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905ac4a-fded-4213-a5e7-3799a5bef870 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:52.885309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:52.885971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.886254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.886595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.887822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:52.889767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:52.891538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 103/206] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:52.898573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.999849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076472-4b2a-42c3-a48e-6a5ed10ce81a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:53.036646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06be205-fd58-472d-b79e-251803b9a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:53.037257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06be205-fd58-472d-b79e-251803b9a785 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:53.037536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06be205-fd58-472d-b79e-251803b9a785 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:53.037832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06be205-fd58-472d-b79e-251803b9a785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:53.038565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06be205-fd58-472d-b79e-251803b9a785 None admin] RBAC: Adding query filter params (role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=8731abd36fdb4e5ca5cfe14ee3093575) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:53.039075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06be205-fd58-472d-b79e-251803b9a785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:53.040358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06be205-fd58-472d-b79e-251803b9a785 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:53.052238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 104/207] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/role_assignments?user.id=8731abd36fdb4e5ca5cfe14ee3093575&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:53.073092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:53.088375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdc2da-f8ed-4fa6-9020-c7319815bae1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:53.159981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457aa787-4c92-4948-a7cd-171e3ea3983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:53.160622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa787-4c92-4948-a7cd-171e3ea3983f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:53.160905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa787-4c92-4948-a7cd-171e3ea3983f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:53.161180 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457aa787-4c92-4948-a7cd-171e3ea3983f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:53.161930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa787-4c92-4948-a7cd-171e3ea3983f None admin] RBAC: Adding query filter params (role.id=3fff797d04c14612b73559493d44d1fd, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491, user.id=e1d77db0bf8f4261a196a1a47603291a) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:53.162242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa787-4c92-4948-a7cd-171e3ea3983f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:53.163818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457aa787-4c92-4948-a7cd-171e3ea3983f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:53.173121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 104/208] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/role_assignments?user.id=e1d77db0bf8f4261a196a1a47603291a&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=3fff797d04c14612b73559493d44d1fd => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:53.976514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7c50-ee3c-4581-b393-22acfde33329 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:53.977026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7c50-ee3c-4581-b393-22acfde33329 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:53.977099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7c50-ee3c-4581-b393-22acfde33329 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:53.978490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 105/209] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:53.984583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c037d-c0b2-4e37-9893-3bca39b7d65b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:53.985034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c037d-c0b2-4e37-9893-3bca39b7d65b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:53.985416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c037d-c0b2-4e37-9893-3bca39b7d65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.004621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-457c037d-c0b2-4e37-9893-3bca39b7d65b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:54.005817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-457c037d-c0b2-4e37-9893-3bca39b7d65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:54.075742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 105/210] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:54.082928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186c47-d38c-4b73-aa05-01b1aa1e1931 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:54.083216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186c47-d38c-4b73-aa05-01b1aa1e1931 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:54.083492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186c47-d38c-4b73-aa05-01b1aa1e1931 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.085104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 106/211] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:54.089976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41dc1-4e9e-4b5f-a0cc-c0a507b062b8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:54.090263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41dc1-4e9e-4b5f-a0cc-c0a507b062b8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:54.090537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41dc1-4e9e-4b5f-a0cc-c0a507b062b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.108513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8eb41dc1-4e9e-4b5f-a0cc-c0a507b062b8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:54.109683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8eb41dc1-4e9e-4b5f-a0cc-c0a507b062b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:54.176285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 106/212] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:54.355283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06186c47-d38c-4b73-aa05-01b1aa1e1931 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:54.430094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:54.450547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41dc1-4e9e-4b5f-a0cc-c0a507b062b8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:54.466220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06186c47-d38c-4b73-aa05-01b1aa1e1931 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:54.466764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:54.506405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:54.507027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:54.507307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:54.507580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.520924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:54.526123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:54.528563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:54.529392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 107/213] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:54.534189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:54.572679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41dc1-4e9e-4b5f-a0cc-c0a507b062b8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:54.615706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:54.616311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:54.616592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:54.616863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.618137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:54.620480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:54.627120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 107/214] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:54.628588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:54.653263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d49c6-3b0d-4fb9-86de-e59ad2fbf3f2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:54.688833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:54.696992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:54.698159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:54.698719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:54.699189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.700117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:54.700640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:54.702827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:54.713408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 108/215] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:54.719338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:54.728082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:54.750061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33515db-d6ba-4abf-b69a-14106f3d09f6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:54.789326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:54.790394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:54.790866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:54.791460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.792437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:54.794264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:54.822319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 108/216] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:37:54 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:37:54.833790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff87c-c980-4139-9636-962b4e54924f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:54.872938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3c1d5-9c15-47e0-bd05-54bf7cf52221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:54.873507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c1d5-9c15-47e0-bd05-54bf7cf52221 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:54.873783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c1d5-9c15-47e0-bd05-54bf7cf52221 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:54.874050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c1d5-9c15-47e0-bd05-54bf7cf52221 None admin] PATH_INFO: `/v3/users/41781d87c9d549b48ea7db079ec0025e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:54.891436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c1d5-9c15-47e0-bd05-54bf7cf52221 None admin] RBAC: Authorizing `identity:get_user(user_id=41781d87c9d549b48ea7db079ec0025e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:54.893048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c1d5-9c15-47e0-bd05-54bf7cf52221 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:54.895372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 109/217] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:54 2022] GET /identity/v3/users/41781d87c9d549b48ea7db079ec0025e => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:54.901211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:55.032198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd8ee8-e4ad-4d30-bc32-e4b74a9a4da1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:55.069519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212f774-aa6c-4151-9636-1c617c9275f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:55.070544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212f774-aa6c-4151-9636-1c617c9275f5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.071063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212f774-aa6c-4151-9636-1c617c9275f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.071522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212f774-aa6c-4151-9636-1c617c9275f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.082347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:55.127113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb53ed-b35b-483f-8f79-bb244e2beb4d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.127599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb53ed-b35b-483f-8f79-bb244e2beb4d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.127599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb53ed-b35b-483f-8f79-bb244e2beb4d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.129756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 110/218] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:55.136499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c8385-95f6-48af-9d7a-467f85ce7e3b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.136770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c8385-95f6-48af-9d7a-467f85ce7e3b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.137041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c8385-95f6-48af-9d7a-467f85ce7e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.157899 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212f774-aa6c-4151-9636-1c617c9275f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:55.159368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212f774-aa6c-4151-9636-1c617c9275f5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:55.161743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c77c8385-95f6-48af-9d7a-467f85ce7e3b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:55.162965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c77c8385-95f6-48af-9d7a-467f85ce7e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:55.216645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:55.242767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 111/219] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:55.275629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 109/220] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:55.281676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c8385-95f6-48af-9d7a-467f85ce7e3b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:55.374142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990142e-7972-43a7-a574-2032d6cc3f0d None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.374910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990142e-7972-43a7-a574-2032d6cc3f0d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.375377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990142e-7972-43a7-a574-2032d6cc3f0d None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.377314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 110/221] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:55.382182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723e4f-b879-441a-bffe-8e7c65a88e4a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.382471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723e4f-b879-441a-bffe-8e7c65a88e4a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.382725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723e4f-b879-441a-bffe-8e7c65a88e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.390159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c8385-95f6-48af-9d7a-467f85ce7e3b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:55.405069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-15723e4f-b879-441a-bffe-8e7c65a88e4a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:55.406251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-15723e4f-b879-441a-bffe-8e7c65a88e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:55.427126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:55.427764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.428107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.428383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.429981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:55.431759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:55.433449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 112/222] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:55 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:55.439290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:55.442755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:55.498966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:55.501083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 111/223] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:37:55.549949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15723e4f-b879-441a-bffe-8e7c65a88e4a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:55.558208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e13ff1-7bd8-4fb3-9482-ea2a9ae1d7ca None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:55.614584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:55.615195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.615528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.615835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.616630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=41781d87c9d549b48ea7db079ec0025e) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:55.616940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:55.618352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:55.632014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 113/224] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:55 2022] GET /identity/v3/role_assignments?user.id=41781d87c9d549b48ea7db079ec0025e&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:55.686103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15723e4f-b879-441a-bffe-8e7c65a88e4a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:55.737136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:55.737792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.738073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.738346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.752952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:55.757931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:55.758838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:37:55.759644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:55.763597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 112/225] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:55.764880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:55.801334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:55.852506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:55.916669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70213195-e37a-4949-91dc-6a5accdc2f04 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:55.932022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:55.963247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:55.963858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.964160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.964428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.965094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:55.965390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:55.966755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:55.975732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 114/226] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:55.981310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:55.997188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b5f6-9c6e-4998-ba00-62487a408116 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.088415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d01448-b42e-47dd-8080-44587a5fc637 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.097156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.098385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.098882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.099352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.110808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:56.121480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:56.122394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:37:56.123227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:56.132111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 113/227] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:37:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:37:56.135783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:56.153709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.155102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.155571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.156085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] PATH_INFO: `/v3/users/8731abd36fdb4e5ca5cfe14ee3093575` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.158269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] RBAC: Authorizing `identity:get_user(user_id=8731abd36fdb4e5ca5cfe14ee3093575)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:56.160050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:56.163276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 115/228] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:55 2022] GET /identity/v3/users/8731abd36fdb4e5ca5cfe14ee3093575 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:56.169649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:56.254172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:56.316345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:56.340895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0ad8-731b-4be8-9c04-c9cc6e245d3a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.366402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a28856-2d82-441f-b6d0-641e1363fe03 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.414304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.416076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.416631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.418485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.419372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:56.421147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:56.422744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:56.438124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.439089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.439586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.440083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.441434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 114/229] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:37:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:56.446370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:56.446370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:56.447942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 116/230] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:56 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:56.448414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:56.453860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:56.613155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70026009-8e40-4a82-bb98-4b553fef8df6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.625529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:56.655304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.656280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.656809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.657393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/8731abd36fdb4e5ca5cfe14ee3093575/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.659659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] RBAC: Authorizing `identity:create_grant(project_id=17313a63ac2f4b9e971725e2dd1f040a, user_id=8731abd36fdb4e5ca5cfe14ee3093575, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:56.661301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:56.661819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be574a6-6870-4c2d-b7db-bb26869dd181 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.681551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 117/231] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:56 2022] PUT /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/8731abd36fdb4e5ca5cfe14ee3093575/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:37:56.710578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.711553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.712064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.712512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] PATH_INFO: `/v3/users/8731abd36fdb4e5ca5cfe14ee3093575` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.713878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] RBAC: Authorizing `identity:get_user(user_id=8731abd36fdb4e5ca5cfe14ee3093575)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:56.715608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:56.717552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 115/232] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:56 2022] GET /identity/v3/users/8731abd36fdb4e5ca5cfe14ee3093575 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:56.728814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:56.853864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcf364-5ecc-4678-9e57-597a153df756 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.892324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab9a5d-78d7-48ed-9f57-af78e9d32429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.893352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab9a5d-78d7-48ed-9f57-af78e9d32429 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.893807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab9a5d-78d7-48ed-9f57-af78e9d32429 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.894341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab9a5d-78d7-48ed-9f57-af78e9d32429 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.895697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab9a5d-78d7-48ed-9f57-af78e9d32429 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:56.897358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab9a5d-78d7-48ed-9f57-af78e9d32429 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:56.899637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 118/233] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:56 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:56.905117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:56.930541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:57.049807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e002042-f2cd-40f2-b523-01a3049d23e2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:57.087857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78e61a-ad1f-4f5b-ad5b-e4b9e1b71c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:57.088939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78e61a-ad1f-4f5b-ad5b-e4b9e1b71c6f None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:57.089418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78e61a-ad1f-4f5b-ad5b-e4b9e1b71c6f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:57.089886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78e61a-ad1f-4f5b-ad5b-e4b9e1b71c6f None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/8731abd36fdb4e5ca5cfe14ee3093575/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:57.092704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78e61a-ad1f-4f5b-ad5b-e4b9e1b71c6f None admin] RBAC: Authorizing `identity:create_grant(project_id=17313a63ac2f4b9e971725e2dd1f040a, user_id=8731abd36fdb4e5ca5cfe14ee3093575, role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:57.094386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78e61a-ad1f-4f5b-ad5b-e4b9e1b71c6f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:57.112402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 116/234] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:56 2022] PUT /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/8731abd36fdb4e5ca5cfe14ee3093575/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:37:57.621461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17035-1f51-4204-973a-d9181683d96d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:57.622520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17035-1f51-4204-973a-d9181683d96d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:57.623037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17035-1f51-4204-973a-d9181683d96d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:57.624765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 119/235] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:57.630643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e2c17-04e9-401c-977a-72dcbf3e3e5a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:57.631322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e2c17-04e9-401c-977a-72dcbf3e3e5a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:57.632111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e2c17-04e9-401c-977a-72dcbf3e3e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:57.661228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c89e2c17-04e9-401c-977a-72dcbf3e3e5a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:57.662979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c89e2c17-04e9-401c-977a-72dcbf3e3e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:57.746529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 117/236] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:57.798039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43a490-053f-4032-870a-68befbcb084f None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:57.798697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43a490-053f-4032-870a-68befbcb084f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:57.799167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43a490-053f-4032-870a-68befbcb084f None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:57.800765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 120/237] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:57.813323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0bcd-d8e6-4cc1-b532-0a01a0ad24a4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:57.813878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0bcd-d8e6-4cc1-b532-0a01a0ad24a4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:57.814347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0bcd-d8e6-4cc1-b532-0a01a0ad24a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:57.838189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-263a0bcd-d8e6-4cc1-b532-0a01a0ad24a4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:57.839722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-263a0bcd-d8e6-4cc1-b532-0a01a0ad24a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:57.842039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:57.932845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 118/238] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:58.133433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43a490-053f-4032-870a-68befbcb084f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:58.177814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:58.231420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0562f4-8c60-42b9-9ef2-27a5b38b319c None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.231957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0562f4-8c60-42b9-9ef2-27a5b38b319c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.231957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0562f4-8c60-42b9-9ef2-27a5b38b319c None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.233278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 119/239] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:58.251109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efcb38-825b-494e-8a7f-da003861871b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.251383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efcb38-825b-494e-8a7f-da003861871b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.251656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efcb38-825b-494e-8a7f-da003861871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.271084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43a490-053f-4032-870a-68befbcb084f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:58.271405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b8efcb38-825b-494e-8a7f-da003861871b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:58.272851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b8efcb38-825b-494e-8a7f-da003861871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:58.319431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:58.320091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.320369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.320639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.333168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:58.338991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:58.339905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:37:58.340749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 121/240] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:58.341417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:58.363604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 120/241] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:58.367986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efcb38-825b-494e-8a7f-da003861871b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:58.472986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:58.512272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:58.517611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980772d-55d2-43d4-bd18-49bc425a431c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:58.527750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efcb38-825b-494e-8a7f-da003861871b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:58.573110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:58.574082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.574602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.575064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.575944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] RBAC: Adding query filter params (name=reader) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:58.576436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:58.578434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:58.587871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 121/242] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:58.593770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:58.610900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:58.611549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.611826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.612118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.616697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:58.618441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:58.620360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 122/243] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:37:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:58.628746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:58.717017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f18261-b29e-4241-934e-00b276798fcb None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:58.753292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:58.760157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:58.761035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.761312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.761587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] PATH_INFO: `/v3/users/867471dbd5434f90bb14aa176395fdcc` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.779634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] RBAC: Authorizing `identity:get_user(user_id=867471dbd5434f90bb14aa176395fdcc)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:58.781207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:58.781746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607e9bc-7a08-4912-a468-3c18026b5691 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:58.783032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 122/244] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:58 2022] GET /identity/v3/users/867471dbd5434f90bb14aa176395fdcc => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:58.783611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:58.821332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:58.822415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.823484 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.824122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.826036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:58.828425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:58.853169 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 123/245] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:37:58 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:37:58.854444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:58.917675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:58.968203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebbd21-a937-4144-aee6-cd11abb0f236 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.041483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.042054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.042333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.042602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.070102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:37:59.075109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:37:59.076057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:37:59.076935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 123/246] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:37:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:37:59.085023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:59.087652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543074a-cd3f-47b6-96d0-8cec0453aa1c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.126526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.127429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.127534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.127783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.152163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:59.214211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:59.215682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:59.218922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291190ae-5c06-4030-b107-8b42e2be5161 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.257843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:59.258987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.259673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.260847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.260847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.261066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] RBAC: Adding query filter params (name=member) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:59.261499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:59.262946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:59.273797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 124/247] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:37:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:59.279200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:59.344329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 124/248] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:59.356593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:59.420089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd2a1-29dc-482e-9b8b-f9c0fdacc95a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.435745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:59.471786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.472481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.472761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.473033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] PATH_INFO: `/v3/users/41781d87c9d549b48ea7db079ec0025e` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.473631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12e839-0224-4a97-ad70-641905d729f9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.474282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] RBAC: Authorizing `identity:get_user(user_id=41781d87c9d549b48ea7db079ec0025e)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:59.476121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:59.478041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 125/249] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:37:59 2022] GET /identity/v3/users/41781d87c9d549b48ea7db079ec0025e => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:37:59.484113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:59.502050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:59.516089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cfc1d-d9f0-4219-a9ee-471287e801b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.516965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cfc1d-d9f0-4219-a9ee-471287e801b7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.517379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cfc1d-d9f0-4219-a9ee-471287e801b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.517809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cfc1d-d9f0-4219-a9ee-471287e801b7 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.519271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cfc1d-d9f0-4219-a9ee-471287e801b7 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:59.520896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cfc1d-d9f0-4219-a9ee-471287e801b7 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:59.523382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 125/250] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:59 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:59.534128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac541c-9f21-46f3-b64e-78ba374bbcf5 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.534411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac541c-9f21-46f3-b64e-78ba374bbcf5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.534681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac541c-9f21-46f3-b64e-78ba374bbcf5 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.536235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 126/251] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:59.536943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac541c-9f21-46f3-b64e-78ba374bbcf5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:59.611135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d816c83-10cd-4472-8d65-82f7f2d15d3a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.646116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ddc07-6898-4322-80e0-8aabaceffca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.646809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ddc07-6898-4322-80e0-8aabaceffca5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.647931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ddc07-6898-4322-80e0-8aabaceffca5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.648239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ddc07-6898-4322-80e0-8aabaceffca5 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.649393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ddc07-6898-4322-80e0-8aabaceffca5 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:59.650847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ddc07-6898-4322-80e0-8aabaceffca5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:59.652813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 126/252] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:37:59 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:59.653953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e76feb-8eb9-41e8-9652-639c1536a24e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.654230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e76feb-8eb9-41e8-9652-639c1536a24e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.654500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e76feb-8eb9-41e8-9652-639c1536a24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.677382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac541c-9f21-46f3-b64e-78ba374bbcf5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.681444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d1e76feb-8eb9-41e8-9652-639c1536a24e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:59.682680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d1e76feb-8eb9-41e8-9652-639c1536a24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:59.715785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e597e7-a8c4-4bd1-9166-a7c27009c8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.716509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e597e7-a8c4-4bd1-9166-a7c27009c8c3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.716879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e597e7-a8c4-4bd1-9166-a7c27009c8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.717148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e597e7-a8c4-4bd1-9166-a7c27009c8c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.717877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e597e7-a8c4-4bd1-9166-a7c27009c8c3 None admin] RBAC: Adding query filter params (role.id=3fff797d04c14612b73559493d44d1fd, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=867471dbd5434f90bb14aa176395fdcc) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:37:59.718331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e597e7-a8c4-4bd1-9166-a7c27009c8c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:59.720582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e597e7-a8c4-4bd1-9166-a7c27009c8c3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:59.726613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 127/253] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:37:59 2022] GET /identity/v3/role_assignments?user.id=867471dbd5434f90bb14aa176395fdcc&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=3fff797d04c14612b73559493d44d1fd => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:37:59.728251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b362f9-e4a4-48b7-80f5-1f10a59b3cf9 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.728835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b362f9-e4a4-48b7-80f5-1f10a59b3cf9 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.729106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b362f9-e4a4-48b7-80f5-1f10a59b3cf9 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.730851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 128/254] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:59.731577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b362f9-e4a4-48b7-80f5-1f10a59b3cf9 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:59.732416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:59.759553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 127/255] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:59.760690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53771e5d-799b-4578-b948-683229efa6b7 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.760973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53771e5d-799b-4578-b948-683229efa6b7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.761243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53771e5d-799b-4578-b948-683229efa6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.793702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-53771e5d-799b-4578-b948-683229efa6b7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:59.795173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-53771e5d-799b-4578-b948-683229efa6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:59.860520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:59.947409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 128/256] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:59.970906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b362f9-e4a4-48b7-80f5-1f10a59b3cf9 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:00.018839 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:00.019579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:00.019857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:00.020196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/41781d87c9d549b48ea7db079ec0025e/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:00.023244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] RBAC: Authorizing `identity:create_grant(project_id=17313a63ac2f4b9e971725e2dd1f040a, user_id=41781d87c9d549b48ea7db079ec0025e, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:00.024797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:00.042129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 129/257] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:37:59 2022] PUT /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/41781d87c9d549b48ea7db079ec0025e/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:00.170739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53771e5d-799b-4578-b948-683229efa6b7 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:00.297710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53771e5d-799b-4578-b948-683229efa6b7 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:00.306245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:00.345976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:00.346625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:00.346952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:00.347220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:00.359590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:00.364550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:00.365509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:00.366337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 129/258] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:00.373627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:00.505965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae5149-c8c7-461b-9ee9-99ce25845afa None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:00.520676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:00.547273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:00.547964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:00.548179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:00.548454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:00.549122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:00.549424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:00.550848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:00.561817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 130/259] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:00.567292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:00.584357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:00.790714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a4ec1-ba66-47d3-94a4-5486922adb49 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:00.818152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6935d2-ac04-447d-9095-b6948f2050dd None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:00.853550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:00.854584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:00.855076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:00.855559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:00.885316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:00.891171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:00.892190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:00.892988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:00.898620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:00.899274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:00.899549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:00.899819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] PATH_INFO: `/v3/users/8731abd36fdb4e5ca5cfe14ee3093575` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:00.900395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 131/260] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:00.901340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] RBAC: Authorizing `identity:get_user(user_id=8731abd36fdb4e5ca5cfe14ee3093575)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:00.902415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:00.906073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:00.908288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 130/261] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:00 2022] GET /identity/v3/users/8731abd36fdb4e5ca5cfe14ee3093575 => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:00.909628 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:00.960833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:00.980056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:01.108562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e438a7-bbf5-47c2-b45f-19e9208d25f5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:01.163972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3467aef-b0e4-419f-b62a-dae965856e1b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:01.189298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:01.189930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:01.190208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:01.190482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:01.191160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:01.191464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:01.192860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:01.211918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 132/262] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:01.217611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:01.253571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:01.254557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:01.255048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:01.255504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:01.368872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:01.374353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c157-afdd-4b08-a8e1-2d5f7dffc0de None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:01.411693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:01.412595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:01.412875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:01.413145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] PATH_INFO: `/v3/users/8731abd36fdb4e5ca5cfe14ee3093575` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:01.413688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:01.414276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] RBAC: Authorizing `identity:get_user(user_id=8731abd36fdb4e5ca5cfe14ee3093575)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:01.415726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:01.416094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:01.418236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 133/263] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:01 2022] GET /identity/v3/users/8731abd36fdb4e5ca5cfe14ee3093575 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:01.425738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:01.480285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:01.608373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 131/264] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:01.614056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:01.638801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45e7fe-03a7-4903-9fc5-0f3325884971 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:01.705889 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:01.706659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:01.706934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:01.707200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:01.773494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:01.826190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746427-701f-4804-9a06-2faca74a4e65 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:01.837930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:01.839418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:01.930318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:01.931322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:01.931791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:01.932271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:01.936566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:01.938238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:01.940102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 132/265] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:01 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:01.949936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:01.958355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:02.033597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 134/266] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:02.039703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:02.183612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:02.186811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187ae33-fd49-4e17-8d70-623a8476c24b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:02.230962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca96270-8dbe-4bf3-b52a-ff4024bf2496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:02.231569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca96270-8dbe-4bf3-b52a-ff4024bf2496 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.231841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca96270-8dbe-4bf3-b52a-ff4024bf2496 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.232151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca96270-8dbe-4bf3-b52a-ff4024bf2496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.232876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca96270-8dbe-4bf3-b52a-ff4024bf2496 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=8731abd36fdb4e5ca5cfe14ee3093575) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:02.233178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca96270-8dbe-4bf3-b52a-ff4024bf2496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:02.234606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca96270-8dbe-4bf3-b52a-ff4024bf2496 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:02.241173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 133/267] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:01 2022] GET /identity/v3/role_assignments?user.id=8731abd36fdb4e5ca5cfe14ee3093575&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:02.242086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918d4fc-aec0-4e55-95ab-2031c649481a None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.242409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918d4fc-aec0-4e55-95ab-2031c649481a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.242627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918d4fc-aec0-4e55-95ab-2031c649481a None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.243796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 134/268] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:02.244930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482126ec-8df9-4d0c-90e5-d68dfc1d1553 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:02.253365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738650ff-091e-41d4-a419-9d81d65c35ec None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.253525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738650ff-091e-41d4-a419-9d81d65c35ec None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.253830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738650ff-091e-41d4-a419-9d81d65c35ec None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.255383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 135/269] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:02.257517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b39be-9cfc-4092-873f-a8a6e5b8ff49 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.257962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b39be-9cfc-4092-873f-a8a6e5b8ff49 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.258063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b39be-9cfc-4092-873f-a8a6e5b8ff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.285506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaee0d7-552b-4c4b-8bd1-75b86576b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:02.286515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaee0d7-552b-4c4b-8bd1-75b86576b36a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.287023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaee0d7-552b-4c4b-8bd1-75b86576b36a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.287561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaee0d7-552b-4c4b-8bd1-75b86576b36a None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.289297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaee0d7-552b-4c4b-8bd1-75b86576b36a None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:02.290929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaee0d7-552b-4c4b-8bd1-75b86576b36a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:02.292978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 135/270] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:02 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:02.294612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3142-ff4a-448d-9155-ca974f90cb9e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.295099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3142-ff4a-448d-9155-ca974f90cb9e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.295555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3142-ff4a-448d-9155-ca974f90cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.297021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-202b39be-9cfc-4092-873f-a8a6e5b8ff49 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:02.298288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-202b39be-9cfc-4092-873f-a8a6e5b8ff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:02.315026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-bd1e3142-ff4a-448d-9155-ca974f90cb9e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:02.316612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bd1e3142-ff4a-448d-9155-ca974f90cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:02.393806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 136/271] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:02.395207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e3142-ff4a-448d-9155-ca974f90cb9e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:02.429926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:02.466979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:02.467167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 136/272] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:02.522412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e3142-ff4a-448d-9155-ca974f90cb9e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:02.567869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:02.568530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.568807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.569073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.569809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] RBAC: Adding query filter params (role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=8731abd36fdb4e5ca5cfe14ee3093575) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:02.570111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:02.571614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:02.578598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 137/273] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:02 2022] GET /identity/v3/role_assignments?user.id=8731abd36fdb4e5ca5cfe14ee3093575&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:02.821916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b39be-9cfc-4092-873f-a8a6e5b8ff49 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:02.858654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:02.960031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b39be-9cfc-4092-873f-a8a6e5b8ff49 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.014661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fd5fe-f840-49db-9775-5e28785f0da6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.021547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.022696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.023917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.024389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.026263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:03.028788 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:03.031498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 137/274] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:03.039914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:03.054578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:03.065558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6b9e5-b0ea-46d5-932f-9085afbd62e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.066253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b9e5-b0ea-46d5-932f-9085afbd62e1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.066529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b9e5-b0ea-46d5-932f-9085afbd62e1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.066796 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6b9e5-b0ea-46d5-932f-9085afbd62e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.079836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6b9e5-b0ea-46d5-932f-9085afbd62e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-9fa6b9e5-b0ea-46d5-932f-9085afbd62e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:03.085820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:03.086729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:03.087593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 138/275] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:03.088358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579071a-64a4-4c4a-b680-d2ba326b820c None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.088961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579071a-64a4-4c4a-b680-d2ba326b820c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.088961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579071a-64a4-4c4a-b680-d2ba326b820c None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.090035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 139/276] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:03.095184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579071a-64a4-4c4a-b680-d2ba326b820c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:03.113614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:03.152324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e187d94-0429-494b-a5b2-bdb09a64d748 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.186093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3434c8-cc5b-423f-9a72-22659e2423fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.186696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3434c8-cc5b-423f-9a72-22659e2423fb None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.186971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3434c8-cc5b-423f-9a72-22659e2423fb None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.187238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3434c8-cc5b-423f-9a72-22659e2423fb None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.187896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3434c8-cc5b-423f-9a72-22659e2423fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:03.189363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3434c8-cc5b-423f-9a72-22659e2423fb None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:03.200733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579071a-64a4-4c4a-b680-d2ba326b820c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.211380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 138/277] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:38:03 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:03.212643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b93f-7a81-47f4-ae02-b2f6e1d8ff17 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.212929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b93f-7a81-47f4-ae02-b2f6e1d8ff17 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.213202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b93f-7a81-47f4-ae02-b2f6e1d8ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.232748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a1f1b93f-7a81-47f4-ae02-b2f6e1d8ff17 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:03.234524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a1f1b93f-7a81-47f4-ae02-b2f6e1d8ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:03.250300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e8330-a7e8-4c45-a934-02d11bc03a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.251048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e8330-a7e8-4c45-a934-02d11bc03a77 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.251325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e8330-a7e8-4c45-a934-02d11bc03a77 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.251598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e8330-a7e8-4c45-a934-02d11bc03a77 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.252314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e8330-a7e8-4c45-a934-02d11bc03a77 None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:03.252620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e8330-a7e8-4c45-a934-02d11bc03a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:03.254036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e8330-a7e8-4c45-a934-02d11bc03a77 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:03.262928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 140/278] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:03.267941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f421ac7-30e3-4a8b-98c9-3855af9c769d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.267941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f421ac7-30e3-4a8b-98c9-3855af9c769d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.267941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f421ac7-30e3-4a8b-98c9-3855af9c769d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.267941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 141/279] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:03.272715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07016-3026-485a-a181-f2a43bac9659 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.272715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07016-3026-485a-a181-f2a43bac9659 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.272715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07016-3026-485a-a181-f2a43bac9659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.293475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-94e07016-3026-485a-a181-f2a43bac9659 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:03.294722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-94e07016-3026-485a-a181-f2a43bac9659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:03.320178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 139/280] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:03.322048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1b93f-7a81-47f4-ae02-b2f6e1d8ff17 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:03.351668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:03.382195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 142/281] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:03.402512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:03.457708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1b93f-7a81-47f4-ae02-b2f6e1d8ff17 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.492879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.493512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.493784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.494050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] PATH_INFO: `/v3/users/0c971b324eaa47c187ae5fdaa026e816` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.509526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] RBAC: Authorizing `identity:get_user(user_id=0c971b324eaa47c187ae5fdaa026e816)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:03.511047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:03.512740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 140/282] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:03 2022] GET /identity/v3/users/0c971b324eaa47c187ae5fdaa026e816 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:03.518766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07016-3026-485a-a181-f2a43bac9659 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:03.603217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:03.631589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07016-3026-485a-a181-f2a43bac9659 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.673772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.674386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.674662 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.674928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.675667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, user.id=0c971b324eaa47c187ae5fdaa026e816, scope.system=all) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:03.676039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:03.677478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:03.683075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 143/283] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Oct 4 08:38:03 2022] GET /identity/v3/role_assignments?user.id=0c971b324eaa47c187ae5fdaa026e816&scope.system=all&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:03.683825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:03.752868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e131c12-63af-44bd-829a-7df5c72ef33d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.755372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:03.770313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:03.809554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.810493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.812624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.813075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.836368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4290db-af90-43eb-ab06-4f0dea1eb74e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:03.848393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:03.853643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:03.854728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:03.855750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 141/284] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:03.865020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:03.886363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:03.887634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:03.888076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:03.888480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:03.913010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:03.918590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:03.919686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:03.920573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 144/285] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:03.927054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:04.023803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be8d72-4e0e-424e-af8e-db10bb30ff80 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.086926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:04.087541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:04.087815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:04.088110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:04.088773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] RBAC: Adding query filter params (name=reader) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:04.089072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:04.090441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:04.098085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a039f-7412-4e13-8dd6-844319b8afbb None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.100893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 142/286] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:04.106400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:04.107324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:04.146601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:04.147215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:04.147486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:04.147749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:04.155910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:04.155910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:04.155910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:04.158939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 145/287] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:04.163601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:04.165907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:04.293616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2f137-f714-4049-86a0-343e4e66b31b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.339222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2f8a-bc3e-4e7a-9020-1774c6a2ce6e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.355197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:04.356217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:04.356814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:04.357394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] PATH_INFO: `/v3/users/867471dbd5434f90bb14aa176395fdcc` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:04.360898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=867471dbd5434f90bb14aa176395fdcc)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:04.362570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:04.365737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 143/288] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:04 2022] GET /identity/v3/users/867471dbd5434f90bb14aa176395fdcc => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:04.370255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:04.375116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:04.375749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:04.376060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:04.376328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] PATH_INFO: `/v3/users/41781d87c9d549b48ea7db079ec0025e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:04.377470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] RBAC: Authorizing `identity:get_user(user_id=41781d87c9d549b48ea7db079ec0025e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:04.378887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:04.380960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 146/289] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:04 2022] GET /identity/v3/users/41781d87c9d549b48ea7db079ec0025e => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:04.392998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:04.437093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:04.505743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a69332-13fe-4eae-9a8f-7d5b9cca8bfc None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.540997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:04.541674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:04.541953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:04.542223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:04.616331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:04.616837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:04.618126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:04.625214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e57a8-be94-4c64-9189-5670598ef3b3 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.668274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:04.720648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 147/290] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:04.726139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:04.752603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:04.753253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:04.753533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:04.753808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:04.755043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:04.756549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:04.758504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 144/291] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:04 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:04.764804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:04.890855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3eba5-4ba4-4edb-ab4c-12714649e4fd None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.953363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:04.955857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a81c51-31da-4842-bb5d-9d243940003d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:04.959291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:04.959590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:04.959867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:04.965133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:04.967963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:04.972815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 148/292] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:04 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:04.981737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:05.000240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:05.030136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1262-9345-47a7-93ea-c4a1e3bb79ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:05.030800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1262-9345-47a7-93ea-c4a1e3bb79ff None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:05.031077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1262-9345-47a7-93ea-c4a1e3bb79ff None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:05.031351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1262-9345-47a7-93ea-c4a1e3bb79ff None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/867471dbd5434f90bb14aa176395fdcc/roles/3fff797d04c14612b73559493d44d1fd` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:05.033974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1262-9345-47a7-93ea-c4a1e3bb79ff None admin] RBAC: Authorizing `identity:create_grant(project_id=17313a63ac2f4b9e971725e2dd1f040a, user_id=867471dbd5434f90bb14aa176395fdcc, role_id=3fff797d04c14612b73559493d44d1fd)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:05.035420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1262-9345-47a7-93ea-c4a1e3bb79ff None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:05.057882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 145/293] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:38:04 2022] PUT /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/users/867471dbd5434f90bb14aa176395fdcc/roles/3fff797d04c14612b73559493d44d1fd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:05.155462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291c703-91df-44b2-aa4f-acb6a7331e1c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:05.218511 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58495cd-c8f8-40ce-94da-d4ca187d2fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:05.219152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58495cd-c8f8-40ce-94da-d4ca187d2fc8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:05.219432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58495cd-c8f8-40ce-94da-d4ca187d2fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:05.219706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58495cd-c8f8-40ce-94da-d4ca187d2fc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:05.220525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58495cd-c8f8-40ce-94da-d4ca187d2fc8 None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=41781d87c9d549b48ea7db079ec0025e) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:05.220835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58495cd-c8f8-40ce-94da-d4ca187d2fc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:05.222299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58495cd-c8f8-40ce-94da-d4ca187d2fc8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:05.229404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 149/294] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:04 2022] GET /identity/v3/role_assignments?user.id=41781d87c9d549b48ea7db079ec0025e&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:05.986798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92dcb0-b151-428d-b474-bacef8e85b42 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:05.987622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92dcb0-b151-428d-b474-bacef8e85b42 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:05.988088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92dcb0-b151-428d-b474-bacef8e85b42 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:05.989948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 146/295] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:05.995762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c7404-a4f3-437d-94c8-4edebb491271 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:05.996960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c7404-a4f3-437d-94c8-4edebb491271 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:05.997424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c7404-a4f3-437d-94c8-4edebb491271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.014148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dc791-02c9-4184-9507-1bbe2de4b109 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.014456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dc791-02c9-4184-9507-1bbe2de4b109 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.014686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dc791-02c9-4184-9507-1bbe2de4b109 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.016276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 147/296] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:38:06.018174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0e6c7404-a4f3-437d-94c8-4edebb491271 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:06.019567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0e6c7404-a4f3-437d-94c8-4edebb491271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:06.022228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36de22-4225-4a71-8692-de14cd232312 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.022723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36de22-4225-4a71-8692-de14cd232312 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.023210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36de22-4225-4a71-8692-de14cd232312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.042316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-cb36de22-4225-4a71-8692-de14cd232312 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:06.043790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-cb36de22-4225-4a71-8692-de14cd232312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:06.066736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:06.084832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:06.100279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 150/297] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:06.127285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 148/298] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:06.184690 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51571f-9dd8-45d5-8f2a-50bfa82ceec7 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.185340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51571f-9dd8-45d5-8f2a-50bfa82ceec7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.185836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51571f-9dd8-45d5-8f2a-50bfa82ceec7 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.187624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 151/299] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:06.192929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea212f-ead8-4757-998b-3730797e7de4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.193210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea212f-ead8-4757-998b-3730797e7de4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.193530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea212f-ead8-4757-998b-3730797e7de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.211614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-96ea212f-ead8-4757-998b-3730797e7de4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:06.214240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-96ea212f-ead8-4757-998b-3730797e7de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:06.278515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 149/300] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:06.371231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51571f-9dd8-45d5-8f2a-50bfa82ceec7 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:06.418770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea212f-ead8-4757-998b-3730797e7de4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:06.490293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51571f-9dd8-45d5-8f2a-50bfa82ceec7 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:06.527277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:06.528043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.528653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:06.528860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.529277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.541862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:06.545946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea212f-ead8-4757-998b-3730797e7de4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:06.547366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:06.548300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:06.549117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:06.553430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 152/301] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:06.554385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:06.585067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:06.585699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.585978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.586234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.587462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:06.588930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:06.595086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 150/302] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:06.595086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:06.616429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:06.661021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c31d7a-e7b7-4dc7-ae73-cb622fec6edb None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:06.694433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d660dc-918e-4d51-ad49-84477ed13a96 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:06.696594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:06.697224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.697496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.697761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.708323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:06.713271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:06.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:06.715005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 153/303] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:06.715746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:06.735299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:06.735929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.736210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.736476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.737137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:06.737434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:06.738799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:06.747836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 151/304] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:06.752927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:06.772059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:06.820131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0bb19-bc39-4a2e-9ed9-e44f26d406c8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:06.837641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:06.857216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:06.857818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.858089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.858206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd207728-3ec6-4bfa-9f05-9c98d4114177 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:06.858363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] PATH_INFO: `/v3/groups` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.858970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:06.860469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:06.879989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 154/305] 10.222.0.44 () {60 vars in 1134 bytes} [Tue Oct 4 08:38:06 2022] POST /identity/v3/groups => generated 229 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:06.879989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:06.913295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:06.913921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:06.914194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:06.914461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:06.915135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] RBAC: Adding query filter params (name=member) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:06.915431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:06.917020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:06.931216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 152/306] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:06.937776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:07.037562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ad2f-614b-4eaf-b34a-5ece6d250b27 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:07.043167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:07.070015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dc2c6-631e-4027-ae85-d52cbae213ec None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:07.091198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:07.092167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:07.092491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:07.092766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] PATH_INFO: `/v3/users/0899f82cde024ae9839e2b7ad9f1486e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:07.111277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] RBAC: Authorizing `identity:get_user(user_id=0899f82cde024ae9839e2b7ad9f1486e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:07.112761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:07.114420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 155/307] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:06 2022] GET /identity/v3/users/0899f82cde024ae9839e2b7ad9f1486e => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:07.115182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:07.115928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:07.116214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:07.116486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] PATH_INFO: `/v3/users/0c971b324eaa47c187ae5fdaa026e816` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:07.117670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] RBAC: Authorizing `identity:get_user(user_id=0c971b324eaa47c187ae5fdaa026e816)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:07.119118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:07.120336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:07.120727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 153/308] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:06 2022] GET /identity/v3/users/0c971b324eaa47c187ae5fdaa026e816 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:07.127065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:07.176256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:07.238239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c8503-6010-489a-84ba-bb4a353de14c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:07.244781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fd43d-b534-491a-9bc2-d6627a138764 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:07.275459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2f5ae-9706-499a-a84c-2d9ff168bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:07.276345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2f5ae-9706-499a-a84c-2d9ff168bcdc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:07.276633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2f5ae-9706-499a-a84c-2d9ff168bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:07.276906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2f5ae-9706-499a-a84c-2d9ff168bcdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:07.278387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2f5ae-9706-499a-a84c-2d9ff168bcdc None admin] RBAC: Adding query filter params (role.id=3fff797d04c14612b73559493d44d1fd, user.id=0899f82cde024ae9839e2b7ad9f1486e, scope.system=all) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:07.278707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2f5ae-9706-499a-a84c-2d9ff168bcdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:07.280226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2f5ae-9706-499a-a84c-2d9ff168bcdc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:07.286889 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 156/309] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Oct 4 08:38:07 2022] GET /identity/v3/role_assignments?user.id=0899f82cde024ae9839e2b7ad9f1486e&scope.system=all&role.id=3fff797d04c14612b73559493d44d1fd => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:07.295246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8def9-b75a-4492-baf0-2c6ff8e289a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:07.296133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8def9-b75a-4492-baf0-2c6ff8e289a6 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:07.296509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8def9-b75a-4492-baf0-2c6ff8e289a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:07.296906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8def9-b75a-4492-baf0-2c6ff8e289a6 None admin] PATH_INFO: `/v3/system/users/0c971b324eaa47c187ae5fdaa026e816/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:07.298645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8def9-b75a-4492-baf0-2c6ff8e289a6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0c971b324eaa47c187ae5fdaa026e816, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:07.300244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8def9-b75a-4492-baf0-2c6ff8e289a6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:07.311466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 154/310] 10.222.0.44 () {58 vars in 1334 bytes} [Tue Oct 4 08:38:07 2022] PUT /identity/v3/system/users/0c971b324eaa47c187ae5fdaa026e816/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:38:07.392487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b74f-d56c-47b1-aa79-57d881b8a026 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:07.392694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b74f-d56c-47b1-aa79-57d881b8a026 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:07.392948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b74f-d56c-47b1-aa79-57d881b8a026 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:07.394363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 157/311] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:07.400471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55c33-0a68-4310-8b27-0945a2f083d9 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:07.400853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55c33-0a68-4310-8b27-0945a2f083d9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:07.401130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55c33-0a68-4310-8b27-0945a2f083d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:07.421086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0ec55c33-0a68-4310-8b27-0945a2f083d9 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:07.422329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0ec55c33-0a68-4310-8b27-0945a2f083d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:07.426760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:07.503870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 155/312] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:07.776107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83b74f-d56c-47b1-aa79-57d881b8a026 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:07.876241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:07.881377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83b74f-d56c-47b1-aa79-57d881b8a026 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:07.915880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:07.916531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:07.916805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:07.917073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:07.927307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:07.932191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:07.933201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:07.934021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 158/313] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:07.941649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55c33-0a68-4310-8b27-0945a2f083d9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:08.051854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55c33-0a68-4310-8b27-0945a2f083d9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:08.086608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:08.087225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.087510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.087783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.088487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:08.088792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:08.090176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:08.099401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 156/314] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:08.105346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:08.210304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4da5-f03a-44b4-9c85-5fa053d6bafe None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:08.245718 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:08.246686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.246960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.247230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] PATH_INFO: `/v3/users/867471dbd5434f90bb14aa176395fdcc` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.248452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] RBAC: Authorizing `identity:get_user(user_id=867471dbd5434f90bb14aa176395fdcc)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:08.249899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:08.251740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 159/315] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:08 2022] GET /identity/v3/users/867471dbd5434f90bb14aa176395fdcc => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:08.257622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:08.260623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:08.366094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e383e-7857-42c4-8bb4-ad01c51b50e5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:08.400078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:08.400940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.401415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.401879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.471337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:08.473130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:08.475208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:08.576082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 157/316] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:08.588623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:08.617546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:08.694830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800de4-cd9a-4257-afe5-7171721881fc None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:08.728211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb3328-3b53-4564-9b93-e4cf01342836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:08.728837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb3328-3b53-4564-9b93-e4cf01342836 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.729112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb3328-3b53-4564-9b93-e4cf01342836 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.729379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb3328-3b53-4564-9b93-e4cf01342836 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.730488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb3328-3b53-4564-9b93-e4cf01342836 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:08.731932 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb3328-3b53-4564-9b93-e4cf01342836 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:08.733580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 160/317] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:08 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:08.739651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:08.842067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711187e7-b174-45b6-944f-f55dee32f4bd None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:08.843525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:08.876679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332151f1-bef4-48c9-9cf3-d7683e6223da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:08.877208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332151f1-bef4-48c9-9cf3-d7683e6223da None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.877485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332151f1-bef4-48c9-9cf3-d7683e6223da None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.877752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332151f1-bef4-48c9-9cf3-d7683e6223da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.878467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332151f1-bef4-48c9-9cf3-d7683e6223da None admin] RBAC: Adding query filter params (role.id=3fff797d04c14612b73559493d44d1fd, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a, user.id=867471dbd5434f90bb14aa176395fdcc) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:08.878767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332151f1-bef4-48c9-9cf3-d7683e6223da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:08.880217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332151f1-bef4-48c9-9cf3-d7683e6223da None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:08.886370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 158/318] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:08 2022] GET /identity/v3/role_assignments?user.id=867471dbd5434f90bb14aa176395fdcc&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=3fff797d04c14612b73559493d44d1fd => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:09.160851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6dd18-6eac-4bbe-9096-03ae247f4d7e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.161341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6dd18-6eac-4bbe-9096-03ae247f4d7e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.161341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6dd18-6eac-4bbe-9096-03ae247f4d7e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.162653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 161/319] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:09.168415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2a5a0-8d15-4b66-b63f-52f6177e8647 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.168853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2a5a0-8d15-4b66-b63f-52f6177e8647 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.168929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2a5a0-8d15-4b66-b63f-52f6177e8647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.187322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0fa2a5a0-8d15-4b66-b63f-52f6177e8647 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:09.188507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0fa2a5a0-8d15-4b66-b63f-52f6177e8647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:09.256870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 159/320] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:09.522625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6dd18-6eac-4bbe-9096-03ae247f4d7e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:09.524565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edaf46-8fac-4255-a38a-626c40325c42 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.525001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edaf46-8fac-4255-a38a-626c40325c42 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.525409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edaf46-8fac-4255-a38a-626c40325c42 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.526927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 160/321] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:38:09.532781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecb683-daa0-4f12-b07a-4fc94e4c21b6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.533343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecb683-daa0-4f12-b07a-4fc94e4c21b6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.533780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecb683-daa0-4f12-b07a-4fc94e4c21b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.552971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b6ecb683-daa0-4f12-b07a-4fc94e4c21b6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:09.554428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b6ecb683-daa0-4f12-b07a-4fc94e4c21b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:09.623372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 161/322] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:09.624700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519aca7-c036-4e5c-a42f-ff277cc9439c None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.624983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519aca7-c036-4e5c-a42f-ff277cc9439c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.625254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519aca7-c036-4e5c-a42f-ff277cc9439c None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.625603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:09.626518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 162/323] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:09.631646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaf4c3-4970-40f5-8321-f0d17b8dd370 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.631944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaf4c3-4970-40f5-8321-f0d17b8dd370 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.632215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaf4c3-4970-40f5-8321-f0d17b8dd370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.634201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6dd18-6eac-4bbe-9096-03ae247f4d7e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:09.649403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-1ffaf4c3-4970-40f5-8321-f0d17b8dd370 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:09.650575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1ffaf4c3-4970-40f5-8321-f0d17b8dd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:09.668437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:09.669062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.669336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.669599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.670663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:09.672224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:09.675491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:09.676051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 162/324] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:09.679392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:09.725182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 163/325] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:38:09.785546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82f975-a468-46d9-a8de-22d2fce15c7b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:09.820149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:09.820749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:09.821024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:09.821290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] PATH_INFO: `/v3/groups` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:09.821918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:09.823334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:09.847914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 163/326] 10.222.0.44 () {60 vars in 1134 bytes} [Tue Oct 4 08:38:09 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:09.891957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaf4c3-4970-40f5-8321-f0d17b8dd370 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:09.992700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:10.009360 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaf4c3-4970-40f5-8321-f0d17b8dd370 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.020810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:10.052027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.052640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.053645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.053645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.066437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:10.071347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:10.073151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:10.074156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 164/327] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:10.078369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:10.102252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:10.113867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805af8fb-a382-4bb4-9eab-94400df78a37 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.154389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.155033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.155331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.155590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.167571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:10.173450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:10.174380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:10.175190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 164/328] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:10.183725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:10.208171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1e684-751d-4da6-8a78-fde429e88622 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.250909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.251534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.251809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.252101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.253329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:10.253637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:10.255036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:10.264327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 165/329] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:10.269605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:10.292104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:10.322149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a674e-9a71-4744-9262-0ee1005b005a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.369040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.369653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.369927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.370197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.371451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:10.371758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:10.374265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:10.384533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:10.387242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 165/330] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:10.393457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:10.414783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a876353-18e8-4e4a-86fc-24e8473ccef5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.452819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.453568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.453892 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.454249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] PATH_INFO: `/v3/users/0899f82cde024ae9839e2b7ad9f1486e` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.455561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] RBAC: Authorizing `identity:get_user(user_id=0899f82cde024ae9839e2b7ad9f1486e)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:10.457298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:10.459283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 166/331] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:10 2022] GET /identity/v3/users/0899f82cde024ae9839e2b7ad9f1486e => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:10.468808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:10.505231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978557f-083f-4e40-8e7d-e407c2a571fc None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.546481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.547416 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.547837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.548284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] PATH_INFO: `/v3/users/0c971b324eaa47c187ae5fdaa026e816` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.549669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] RBAC: Authorizing `identity:get_user(user_id=0c971b324eaa47c187ae5fdaa026e816)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:10.553454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:10.555957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 166/332] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:10 2022] GET /identity/v3/users/0c971b324eaa47c187ae5fdaa026e816 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:10.562140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:10.579926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:10.592798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab0d94-f19a-44a9-98bd-56dfbd5d5bbc None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.637933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa1256-b856-415b-8d1b-21c8119f0119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.639123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa1256-b856-415b-8d1b-21c8119f0119 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.639937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa1256-b856-415b-8d1b-21c8119f0119 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.640353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa1256-b856-415b-8d1b-21c8119f0119 None admin] PATH_INFO: `/v3/system/users/0899f82cde024ae9839e2b7ad9f1486e/roles/3fff797d04c14612b73559493d44d1fd` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.642257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa1256-b856-415b-8d1b-21c8119f0119 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0899f82cde024ae9839e2b7ad9f1486e, role_id=3fff797d04c14612b73559493d44d1fd)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:10.644713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa1256-b856-415b-8d1b-21c8119f0119 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:10.654922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 167/333] 10.222.0.44 () {58 vars in 1334 bytes} [Tue Oct 4 08:38:10 2022] PUT /identity/v3/system/users/0899f82cde024ae9839e2b7ad9f1486e/roles/3fff797d04c14612b73559493d44d1fd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:38:10.693734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea081f0-0d52-4406-8ffe-c3edd1815384 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:10.740724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997fcc-102a-4946-ba2c-e1357225d401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:10.741353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997fcc-102a-4946-ba2c-e1357225d401 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:10.741628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997fcc-102a-4946-ba2c-e1357225d401 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:10.741895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997fcc-102a-4946-ba2c-e1357225d401 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:10.742644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997fcc-102a-4946-ba2c-e1357225d401 None admin] RBAC: Adding query filter params (role.id=503f8b8f2dc44b52a8f6901954713e7d, user.id=0c971b324eaa47c187ae5fdaa026e816, scope.system=all) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:10.742945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997fcc-102a-4946-ba2c-e1357225d401 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:10.744428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997fcc-102a-4946-ba2c-e1357225d401 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:10.759803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 167/334] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Oct 4 08:38:10 2022] GET /identity/v3/role_assignments?user.id=0c971b324eaa47c187ae5fdaa026e816&scope.system=all&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:12.262187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b3b5e-81f3-4045-9be4-99c0e3f9d901 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:12.262665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b3b5e-81f3-4045-9be4-99c0e3f9d901 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:12.262665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b3b5e-81f3-4045-9be4-99c0e3f9d901 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:12.266153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 168/335] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:12.272089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b40da-efba-4217-ba0f-47e74e4f3ce2 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:12.272929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b40da-efba-4217-ba0f-47e74e4f3ce2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:12.273392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b40da-efba-4217-ba0f-47e74e4f3ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:12.300955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-eb9b40da-efba-4217-ba0f-47e74e4f3ce2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:12.302499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-eb9b40da-efba-4217-ba0f-47e74e4f3ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:12.381544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:12.391351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 168/336] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:12.886470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f105-68cc-4bca-ad5f-e78458a02087 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:12.886845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f105-68cc-4bca-ad5f-e78458a02087 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:12.887036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f105-68cc-4bca-ad5f-e78458a02087 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:12.888661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 169/337] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:12.893739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7817e3-96ca-405d-ac86-944265a1fd04 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:12.894029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7817e3-96ca-405d-ac86-944265a1fd04 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:12.894311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7817e3-96ca-405d-ac86-944265a1fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:12.914803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9c7817e3-96ca-405d-ac86-944265a1fd04 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:12.915239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f105-68cc-4bca-ad5f-e78458a02087 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:12.916130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9c7817e3-96ca-405d-ac86-944265a1fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:12.934673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:13.038709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 169/338] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:13.113537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f105-68cc-4bca-ad5f-e78458a02087 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:13.156720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:13.157287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.157568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.157842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.168864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:13.173694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:13.174560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:13.175757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 170/339] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:13.182573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7817e3-96ca-405d-ac86-944265a1fd04 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:13.322316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7817e3-96ca-405d-ac86-944265a1fd04 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:13.357679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:13.364250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:13.364864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.365170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.365440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.366108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:13.366408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:13.367842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:13.378745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 170/340] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:13.385200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:13.478584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:13.504837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:13.520417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f744-de58-4ab5-a03d-8eaa40a201c9 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:13.582866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357048ba-0a2b-4c88-af12-d459dd5d73fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:13.583501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357048ba-0a2b-4c88-af12-d459dd5d73fe None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.583811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357048ba-0a2b-4c88-af12-d459dd5d73fe None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.584101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357048ba-0a2b-4c88-af12-d459dd5d73fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.694780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357048ba-0a2b-4c88-af12-d459dd5d73fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:13.696303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357048ba-0a2b-4c88-af12-d459dd5d73fe None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:13.762950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4b0f6-46f1-4997-a8ba-264a6ca912a8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:13.808366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:13.844477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:13.845142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.845422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.845695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.873868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:13.876989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 171/341] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:13.877440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba7f6f-ede6-44a7-9536-72e762fcacc0 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.877727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba7f6f-ede6-44a7-9536-72e762fcacc0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.878002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba7f6f-ede6-44a7-9536-72e762fcacc0 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.879331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 172/342] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:13.883284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:13.885709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:13.886518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:13.889867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864ff0e-6597-4f9d-8b01-ac267582b6e0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.889867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864ff0e-6597-4f9d-8b01-ac267582b6e0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.890048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864ff0e-6597-4f9d-8b01-ac267582b6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.890284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 171/343] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:13.892671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:13.922704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8864ff0e-6597-4f9d-8b01-ac267582b6e0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:13.924270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8864ff0e-6597-4f9d-8b01-ac267582b6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:13.996179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:14.023625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 173/344] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:14.024416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864ff0e-6597-4f9d-8b01-ac267582b6e0 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:14.065633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ca7ba-1d10-4748-82cb-5768db5129f4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:14.109548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f67f37-a29d-4b88-bbf0-910dca7c3bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:14.110558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f67f37-a29d-4b88-bbf0-910dca7c3bbe None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.111035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f67f37-a29d-4b88-bbf0-910dca7c3bbe None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.111520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f67f37-a29d-4b88-bbf0-910dca7c3bbe None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.114051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f67f37-a29d-4b88-bbf0-910dca7c3bbe None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:14.115537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f67f37-a29d-4b88-bbf0-910dca7c3bbe None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:14.122045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 172/345] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:13 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:14.123301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f67944-94a7-448c-8c0b-9b0a347606bb None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.123580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f67944-94a7-448c-8c0b-9b0a347606bb None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.123849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f67944-94a7-448c-8c0b-9b0a347606bb None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.125268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 173/346] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:14.126153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a95f-5d9d-4324-9e73-5adb61f10865 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.126428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a95f-5d9d-4324-9e73-5adb61f10865 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.126693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a95f-5d9d-4324-9e73-5adb61f10865 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.131503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 174/347] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:14.131503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896a95f-5d9d-4324-9e73-5adb61f10865 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:14.163739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:14.167396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864ff0e-6597-4f9d-8b01-ac267582b6e0 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:14.220479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075cb17-3892-491f-a107-7e593206b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:14.221813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075cb17-3892-491f-a107-7e593206b20f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.222629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075cb17-3892-491f-a107-7e593206b20f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.223263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075cb17-3892-491f-a107-7e593206b20f None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.224290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075cb17-3892-491f-a107-7e593206b20f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:14.225011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075cb17-3892-491f-a107-7e593206b20f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:14.227237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075cb17-3892-491f-a107-7e593206b20f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:14.240386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 174/348] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:14.242712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28dcc6-94b7-4805-89c8-ca634c28dc66 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.243353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28dcc6-94b7-4805-89c8-ca634c28dc66 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.244114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28dcc6-94b7-4805-89c8-ca634c28dc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.259506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896a95f-5d9d-4324-9e73-5adb61f10865 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:14.267485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ee28dcc6-94b7-4805-89c8-ca634c28dc66 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:14.268677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ee28dcc6-94b7-4805-89c8-ca634c28dc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:14.292239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:14.310150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539cf13-812b-492e-b643-148d9cec92bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:14.310939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539cf13-812b-492e-b643-148d9cec92bf None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.311313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539cf13-812b-492e-b643-148d9cec92bf None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.311685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539cf13-812b-492e-b643-148d9cec92bf None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.318893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539cf13-812b-492e-b643-148d9cec92bf None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:14.320528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539cf13-812b-492e-b643-148d9cec92bf None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:14.322479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 175/349] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:14 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:14.323710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565cee-eb90-4b40-bd59-ebb3d0cfca52 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.324146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565cee-eb90-4b40-bd59-ebb3d0cfca52 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.324615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565cee-eb90-4b40-bd59-ebb3d0cfca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.346318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-59565cee-eb90-4b40-bd59-ebb3d0cfca52 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:14.347505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-59565cee-eb90-4b40-bd59-ebb3d0cfca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:14.362143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 175/350] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:14.363750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28dcc6-94b7-4805-89c8-ca634c28dc66 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:14.420744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 176/351] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:14.421710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565cee-eb90-4b40-bd59-ebb3d0cfca52 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:14.530221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:14.547187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28dcc6-94b7-4805-89c8-ca634c28dc66 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:14.589920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565cee-eb90-4b40-bd59-ebb3d0cfca52 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:14.611338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:14.612043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.612320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.612591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.617786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:14.651424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:14.652078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.652362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.652840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.653728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:14.654042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:14.657309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:14.664753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 177/352] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:14 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:14.665674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:14.770289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:14.771841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:14.820644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68d87-3f77-4cb2-a9ff-e94cf27dbe3d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:14.879329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:14.880043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:14.880420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:14.880772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:14.892632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:14.897441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:14.904672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:14.905660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:14.906486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 178/353] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:14.907172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 176/354] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:14.907244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:14.908047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:14.968516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:15.032404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732202-b31b-4cac-9f8b-687000cf0338 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.052141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beedb73-3358-43bc-adb8-5fb081299b9b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.069039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.069700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.069978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.070246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.083607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:15.088616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:15.089599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:15.090387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 177/355] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:15.091051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.106873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.108102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.109006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.109438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.122881 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:15.129167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:15.130084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:38:15.130945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:15.133373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 179/356] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:15.134492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.152093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:15.212250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90b6e7-1975-43d6-9590-0a5cd10fc562 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.259088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.259733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.260066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.260335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.261467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:15.262918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:15.264658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502ab52-07b8-456b-ad70-2164046e4170 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.264937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 178/357] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:15.265769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.266551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:15.314222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.315391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.316303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.316937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.317911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:15.318430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:15.319815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:15.329481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 180/358] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:15.330324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.387477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b785c-a8f3-45c6-9ea7-e6f84cbd58f4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.412567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:15.432498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167d1ae-b790-4a7d-b90f-2ec798b60df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.433122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167d1ae-b790-4a7d-b90f-2ec798b60df3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.433404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167d1ae-b790-4a7d-b90f-2ec798b60df3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.433676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167d1ae-b790-4a7d-b90f-2ec798b60df3 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.434345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167d1ae-b790-4a7d-b90f-2ec798b60df3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:15.434648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167d1ae-b790-4a7d-b90f-2ec798b60df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:15.436073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167d1ae-b790-4a7d-b90f-2ec798b60df3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:15.444954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 179/359] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:15.446329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b339c-f068-4dd2-8a9d-cdb75180d2a6 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.446652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b339c-f068-4dd2-8a9d-cdb75180d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.446930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b339c-f068-4dd2-8a9d-cdb75180d2a6 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.448467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 180/360] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:15.449365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b339c-f068-4dd2-8a9d-cdb75180d2a6 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.462606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfc0d7-a2d3-44b2-8ea3-d58e5354bf4d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.504124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.504736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.505009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.505275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.505975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:15.506272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:15.507624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:15.518019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 181/361] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:15.522993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.547004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:15.577949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b339c-f068-4dd2-8a9d-cdb75180d2a6 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.620475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ecd612-309d-4100-a434-2f2d63cf284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.621432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecd612-309d-4100-a434-2f2d63cf284a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.623620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecd612-309d-4100-a434-2f2d63cf284a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.623941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecd612-309d-4100-a434-2f2d63cf284a None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.625094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecd612-309d-4100-a434-2f2d63cf284a None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:15.626547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecd612-309d-4100-a434-2f2d63cf284a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:15.629426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 181/362] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:15.630605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db79d1-8923-46ce-9cde-3fdc5ea16e82 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.631156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db79d1-8923-46ce-9cde-3fdc5ea16e82 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.631705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db79d1-8923-46ce-9cde-3fdc5ea16e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.639021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3642ca-99a8-4364-b606-3016121b8e16 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.654918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e1db79d1-8923-46ce-9cde-3fdc5ea16e82 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:15.656436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e1db79d1-8923-46ce-9cde-3fdc5ea16e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:15.682076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.683164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.683758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.684300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.712233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:15.736314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 182/363] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:15.738078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db79d1-8923-46ce-9cde-3fdc5ea16e82 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.774868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:15.776360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:15.804568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:15.869649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db79d1-8923-46ce-9cde-3fdc5ea16e82 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:15.909169 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 182/364] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:15.910043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:15.922385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:15.923790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:15.924287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:15.924746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:15.989353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.015666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.017472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.053122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42e217-476c-4a71-8515-43137e98dd25 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.078960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.097897 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.098543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.098814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.099079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] PATH_INFO: `/v3/users/0899f82cde024ae9839e2b7ad9f1486e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.100542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] RBAC: Authorizing `identity:get_user(user_id=0899f82cde024ae9839e2b7ad9f1486e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.102437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.104991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 183/365] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/users/0899f82cde024ae9839e2b7ad9f1486e => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.105790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.151164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 183/366] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:16.156647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.236562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706241fb-6303-4b76-ab8d-b8547f30b5ab None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.254994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.282464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651ac7-4f37-4aa6-8a5b-374d1714bc13 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.288608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.289227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.289505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.289774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.290512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:16.290816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.293131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.299866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 184/367] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.304185 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.333919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.334572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.334843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.335106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.337150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.339220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.340989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 184/368] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.341854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.378936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.438832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb8c87-0f47-4ecd-af40-48ec819a9b4f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.485979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a382c0-319b-4f6f-8b87-492098c7d006 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.486946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.487532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.487809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.488110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.505201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:16.511202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:16.512256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:16.513098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 185/369] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:16.513807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.538403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.538942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.539442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.539553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.540243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] RBAC: Adding query filter params (role.id=3fff797d04c14612b73559493d44d1fd, user.id=0899f82cde024ae9839e2b7ad9f1486e, scope.system=all) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:16.540559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.542007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.565353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 185/370] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/role_assignments?user.id=0899f82cde024ae9839e2b7ad9f1486e&scope.system=all&role.id=3fff797d04c14612b73559493d44d1fd => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.566270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.584138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.651913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0a753-86da-4bd1-9b38-89fd7c06d716 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.668526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.703107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.704377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.704894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.705360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.710131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.712044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.714162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 186/371] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.715162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.742512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f2cde-c57b-498e-84f3-091c807cc9d6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.779067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.779855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.779988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.780259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.781362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.782809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.790269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 186/372] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.790269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.865659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.874572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e3cfe-cb72-4c43-898f-b13cbe5da626 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.891143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762996a-d1f3-4490-b398-2d29ef97c637 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:16.923234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.924331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.924820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.925294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.926345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:16.926983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.928960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.929096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:16.929621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.929894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.930158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.932319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:16.933010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:16.934682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 187/373] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.935717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:16.942767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 187/374] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:16.945808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:17.005172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.052926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442336c-4be7-409c-b00a-ca2e53e56547 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:17.071678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726533d-498d-45fb-9246-2d9314ee1ddc None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:17.087777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:17.088468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.088745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.089025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.089725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:17.090026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:17.091442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:17.098193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 188/375] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:17.099087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:17.108801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.116924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fa341-4696-4429-a6af-e416dcc04bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:17.117899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fa341-4696-4429-a6af-e416dcc04bdd None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.118374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fa341-4696-4429-a6af-e416dcc04bdd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.118985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fa341-4696-4429-a6af-e416dcc04bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.120113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fa341-4696-4429-a6af-e416dcc04bdd None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:17.120666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fa341-4696-4429-a6af-e416dcc04bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:17.122673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fa341-4696-4429-a6af-e416dcc04bdd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:17.129850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 188/376] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:16 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:17.212322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957b0bd-d935-4ad4-a220-ea3041713886 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:17.243050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e30ff7-4eb5-481e-b359-5ea8e30fa3b0 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.243709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e30ff7-4eb5-481e-b359-5ea8e30fa3b0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.244220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e30ff7-4eb5-481e-b359-5ea8e30fa3b0 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.246683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 189/377] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:17.252159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175b16-bf89-41a9-ae17-cf394e8c1fc1 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.252830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175b16-bf89-41a9-ae17-cf394e8c1fc1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.253184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:17.253561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175b16-bf89-41a9-ae17-cf394e8c1fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.253809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.254084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.254351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.291618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9c175b16-bf89-41a9-ae17-cf394e8c1fc1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:17.293721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9c175b16-bf89-41a9-ae17-cf394e8c1fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:17.332095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:17.332095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:17.340745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.399784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 190/378] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:17.448658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 189/379] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:17.454721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175b16-bf89-41a9-ae17-cf394e8c1fc1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:17.500347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.565146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175b16-bf89-41a9-ae17-cf394e8c1fc1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:17.604134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:17.604783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.605060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.605328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] PATH_INFO: `/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.613212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b22621efa4e2c8d25ad2792cb5242)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:17.614047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:17.616150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 191/380] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:17 2022] GET /identity/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:17.621634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:17.756387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a303d-7c00-4fe1-af6d-45bb7c55d04c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:17.764642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.795765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:17.796426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.796699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.796963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] PATH_INFO: `/v3/groups/e12f441a1bcc4b0b842550c958fe0a3a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.808223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] RBAC: Authorizing `identity:get_group(group_id=e12f441a1bcc4b0b842550c958fe0a3a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:17.809728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:17.811725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 190/381] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:17 2022] GET /identity/v3/groups/e12f441a1bcc4b0b842550c958fe0a3a => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:17.817170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:17.873850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:17.995324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bed61-4537-4b2f-aa76-94ad266d718b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:18.012342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:18.037909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73e07-6856-4196-91c6-590fd56e3b2c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:18.044596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:18.045209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:18.045481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:18.045747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:18.046472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] RBAC: Adding query filter params (group.id=e12f441a1bcc4b0b842550c958fe0a3a, role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=5d6b22621efa4e2c8d25ad2792cb5242) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:18.046774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:18.049184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:18.055454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 192/382] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:17 2022] GET /identity/v3/role_assignments?group.id=e12f441a1bcc4b0b842550c958fe0a3a&scope.project.id=5d6b22621efa4e2c8d25ad2792cb5242&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:18.111485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:18.112090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:18.112375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:18.112644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:18.131316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:18.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:18.138477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:18.139728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 191/383] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:18.146927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:18.285974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ceef8-e373-4ec4-b347-43aa214ba1bf None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:18.329275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10317556-b485-41af-b092-73b8ce65309e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:18.329920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10317556-b485-41af-b092-73b8ce65309e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:18.330196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10317556-b485-41af-b092-73b8ce65309e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:18.330468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10317556-b485-41af-b092-73b8ce65309e None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:18.331165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10317556-b485-41af-b092-73b8ce65309e None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:18.331468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10317556-b485-41af-b092-73b8ce65309e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:18.332962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10317556-b485-41af-b092-73b8ce65309e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:18.346783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 193/384] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:18.352970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:18.397625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:18.484061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193206d-efec-4e2d-9083-cd92f9556610 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:18.530047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:18.530694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:18.530972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:18.531240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:18.532461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:18.533921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:18.535501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 192/385] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:18 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:18.541610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10317556-b485-41af-b092-73b8ce65309e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:18.590745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:18.673578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10317556-b485-41af-b092-73b8ce65309e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:18.714901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634cda0-308c-4cf8-b692-4e03bdda73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:18.715540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634cda0-308c-4cf8-b692-4e03bdda73b8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:18.715816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634cda0-308c-4cf8-b692-4e03bdda73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:18.716116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634cda0-308c-4cf8-b692-4e03bdda73b8 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:18.717212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634cda0-308c-4cf8-b692-4e03bdda73b8 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:18.718638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634cda0-308c-4cf8-b692-4e03bdda73b8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:18.720576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 194/386] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:18 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:18.726471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:18.858754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:18.862954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f85d26-a0e7-4e2d-965b-b4bec0dd4110 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:18.901337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981455f-e27e-4b03-a8b0-c7a23f9021e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:18.902005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981455f-e27e-4b03-a8b0-c7a23f9021e6 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:18.902282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981455f-e27e-4b03-a8b0-c7a23f9021e6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:18.902545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981455f-e27e-4b03-a8b0-c7a23f9021e6 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:18.909165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981455f-e27e-4b03-a8b0-c7a23f9021e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec27f0f03e8249ff8c379f6356fc3491, group_id=8e555b8a93054e6d937c5c1cde270ab9, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:18.910612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981455f-e27e-4b03-a8b0-c7a23f9021e6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:18.924750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 193/387] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Oct 4 08:38:18 2022] PUT /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:19.215624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a9c0-6cf5-452c-9baf-5717314a2616 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:19.215979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a9c0-6cf5-452c-9baf-5717314a2616 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:19.216093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a9c0-6cf5-452c-9baf-5717314a2616 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:19.218193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 195/388] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:19.228808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cf627-1754-4efd-af5c-d395e2203731 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:19.228808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cf627-1754-4efd-af5c-d395e2203731 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:19.228808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cf627-1754-4efd-af5c-d395e2203731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:19.246219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-459cf627-1754-4efd-af5c-d395e2203731 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:19.247576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-459cf627-1754-4efd-af5c-d395e2203731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:19.323329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 194/389] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:19.752647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012a9c0-6cf5-452c-9baf-5717314a2616 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:19.870118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:19.875459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012a9c0-6cf5-452c-9baf-5717314a2616 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:19.887572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ac8f-a4e2-4a4f-b108-60ba7dc54ff5 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:19.888088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ac8f-a4e2-4a4f-b108-60ba7dc54ff5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:19.888196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ac8f-a4e2-4a4f-b108-60ba7dc54ff5 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:19.889601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 195/390] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:19.898578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c44882-4c8d-4b7b-a993-601b116dfc73 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:19.898578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c44882-4c8d-4b7b-a993-601b116dfc73 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:19.898578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c44882-4c8d-4b7b-a993-601b116dfc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:19.916832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-43c44882-4c8d-4b7b-a993-601b116dfc73 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:19.918617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-43c44882-4c8d-4b7b-a993-601b116dfc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:19.946671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:19.947332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:19.947606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:19.947875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:19.975505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:19.980582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:19.981471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:19.982264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:19.983548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 196/391] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:19.988976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.063910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 196/392] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:38:20.076630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407f8bd-b92d-4ed1-be43-66c5d714e27a None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.076937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407f8bd-b92d-4ed1-be43-66c5d714e27a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.077185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407f8bd-b92d-4ed1-be43-66c5d714e27a None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.078852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 197/393] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:20.083942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec60c-3adc-42bf-a48e-5fce2334fc1c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.084221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec60c-3adc-42bf-a48e-5fce2334fc1c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.084488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeec60c-3adc-42bf-a48e-5fce2334fc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.102834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2aeec60c-3adc-42bf-a48e-5fce2334fc1c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:20.104140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2aeec60c-3adc-42bf-a48e-5fce2334fc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:20.123430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:20.185473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297c84-b6b1-46a9-a1d1-e75d1f78f1e7 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.221762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 198/394] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:20.242268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:20.242878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.243149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.243412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.244097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:20.244402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:20.246071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:20.259396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 197/395] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:20.264758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeec60c-3adc-42bf-a48e-5fce2334fc1c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.369098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.413339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:20.414760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeec60c-3adc-42bf-a48e-5fce2334fc1c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.474820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:20.475452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.475724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.476012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.477697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:20.479139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:20.480747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 199/396] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:20 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:20.490318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.520257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:20.544510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6019009-7f64-4289-b378-9b5fe0381a32 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.584052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:20.584965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.585395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.585803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.605380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:20.611429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:20.616915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:20.618260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:20.625729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46202cfc-2f44-450a-acb3-2837df9af120 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.626993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 198/397] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:20.628043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.664347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:20.665324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.665883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.666434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.667946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:20.669864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:20.672786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 200/398] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:20 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:20.673775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.726753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:20.749705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7abc-b1c0-4d55-9fe8-0611c940293a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.780931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:20.787393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:20.788061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.788337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.788609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.798846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7310ac-2424-48b2-8220-54278b3cd26c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.799112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:20.804940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:20.805857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:20.806676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 199/399] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:20.807375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.836384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:20.837008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.837282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.837549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.838317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] RBAC: Adding query filter params (name=member) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:20.838770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:20.840195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:20.854842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 201/400] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:20.855978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.913462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2c4a4-4453-4e31-8c2f-9c979c7ceeed None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.948256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:20.951455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:20.951455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:20.951455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:20.951929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec27f0f03e8249ff8c379f6356fc3491, group_id=8e555b8a93054e6d937c5c1cde270ab9, role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:20.953395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:20.969206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d2cfe-654c-4f6c-927a-2c617c900b08 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:20.969696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 200/401] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Oct 4 08:38:20 2022] PUT /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:20.970321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:20.974045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:21.006998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17de94-7676-4f1b-b606-2b27a228e52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.007622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17de94-7676-4f1b-b606-2b27a228e52c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.007910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17de94-7676-4f1b-b606-2b27a228e52c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.008194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17de94-7676-4f1b-b606-2b27a228e52c None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.009015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17de94-7676-4f1b-b606-2b27a228e52c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:21.009361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17de94-7676-4f1b-b606-2b27a228e52c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:21.010874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17de94-7676-4f1b-b606-2b27a228e52c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:21.020113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 202/402] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:21.021809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d7aab-4fad-4234-bf26-8ea99182c39b None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.022093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d7aab-4fad-4234-bf26-8ea99182c39b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.022364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d7aab-4fad-4234-bf26-8ea99182c39b None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.023826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 203/403] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:21.029019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d7aab-4fad-4234-bf26-8ea99182c39b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:21.047222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:21.094382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab7bf-504b-442e-a4c7-829fdf8f3b00 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:21.132668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa186d6-52ce-49e6-b6f6-67898617f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.133381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa186d6-52ce-49e6-b6f6-67898617f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.133663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa186d6-52ce-49e6-b6f6-67898617f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.133995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa186d6-52ce-49e6-b6f6-67898617f4b6 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.135204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa186d6-52ce-49e6-b6f6-67898617f4b6 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:21.136680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa186d6-52ce-49e6-b6f6-67898617f4b6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:21.143313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 201/404] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:20 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:21.145273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35cb90-054a-4184-8f6f-6d4406ffd9fd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.145751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35cb90-054a-4184-8f6f-6d4406ffd9fd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.146030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35cb90-054a-4184-8f6f-6d4406ffd9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.166644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d7aab-4fad-4234-bf26-8ea99182c39b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:21.167898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5f35cb90-054a-4184-8f6f-6d4406ffd9fd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:21.169146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5f35cb90-054a-4184-8f6f-6d4406ffd9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:21.216886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.217971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.218249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.218520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.220106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:21.221875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:21.224606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 204/405] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:21 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:21.225638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:21.241482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:21.248226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 202/406] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:21.249364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35cb90-054a-4184-8f6f-6d4406ffd9fd None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:21.373589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:21.398918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb686b1-20c6-4036-82d3-f3be755ed64f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:21.433159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.434127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.434876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.435354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.436702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:21.438421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:21.440582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 205/407] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:21 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:21.443690 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35cb90-054a-4184-8f6f-6d4406ffd9fd None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:21.446322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:21.502666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.503860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.505127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.506165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.509464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:21.511087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:21.518616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 203/408] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:21 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:21.519739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:21.614881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766ee72-e6a7-48b6-9dd1-92f819fe7d2b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:21.629499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:21.660305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.661473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.662027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.662764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.665567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] RBAC: Authorizing `identity:create_grant(project_id=17313a63ac2f4b9e971725e2dd1f040a, group_id=8e555b8a93054e6d937c5c1cde270ab9, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:21.667627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:21.672128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5eec7-ae67-442b-93f3-c210e5d135bb None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:21.686007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 206/409] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Oct 4 08:38:21 2022] PUT /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:21.687612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:21.692654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:21.717022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b72628-d3bc-4097-b67c-a191ee904875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.718238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b72628-d3bc-4097-b67c-a191ee904875 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.718641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b72628-d3bc-4097-b67c-a191ee904875 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.719019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b72628-d3bc-4097-b67c-a191ee904875 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.721512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b72628-d3bc-4097-b67c-a191ee904875 None admin] RBAC: Authorizing `identity:create_grant(project_id=17313a63ac2f4b9e971725e2dd1f040a, group_id=8e555b8a93054e6d937c5c1cde270ab9, role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:21.723914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b72628-d3bc-4097-b67c-a191ee904875 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:21.743641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 204/410] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Oct 4 08:38:21 2022] PUT /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a/groups/8e555b8a93054e6d937c5c1cde270ab9/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:21.745532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29822c-d099-4880-a23b-21ed5f4f1f64 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.746375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29822c-d099-4880-a23b-21ed5f4f1f64 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.746810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29822c-d099-4880-a23b-21ed5f4f1f64 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.748628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 205/411] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:21.757547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0072e2d-aa54-4144-aa7f-51c5d388b818 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.758150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0072e2d-aa54-4144-aa7f-51c5d388b818 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.758649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0072e2d-aa54-4144-aa7f-51c5d388b818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.803358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b0072e2d-aa54-4144-aa7f-51c5d388b818 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:21.810225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b0072e2d-aa54-4144-aa7f-51c5d388b818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:21.867857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20976821-8e8a-4ad1-8baf-156e81a35b70 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:21.925528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:21.926269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.926546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.926879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.937434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 206/412] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:21.951919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:21.952114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:21.952978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:21.953794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 207/413] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:21.960818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0072e2d-aa54-4144-aa7f-51c5d388b818 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.030503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:22.080848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0072e2d-aa54-4144-aa7f-51c5d388b818 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:22.121778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:22.122401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:22.122679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:22.122948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:22.123616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:22.123936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:22.125383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:22.138474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 207/414] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:22.139719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.226334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:22.269983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a74ec-dc8d-439b-bbe2-7deefbb92549 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:22.304044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.306160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:22.306800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:22.307090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:22.307335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] PATH_INFO: `/v3/groups/e12f441a1bcc4b0b842550c958fe0a3a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:22.308550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] RBAC: Authorizing `identity:get_group(group_id=e12f441a1bcc4b0b842550c958fe0a3a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:22.310286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:22.311906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 208/415] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:22 2022] GET /identity/v3/groups/e12f441a1bcc4b0b842550c958fe0a3a => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:22.318449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.466153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f008e-8825-4786-a492-dc74f6cd8825 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:22.502782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe461b4-e9ac-435d-a766-d64c8d279609 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:22.504301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:22.527818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:22.528481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:22.528751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:22.529012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:22.550680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:22.556264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:22.557182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:22.557995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:22.563195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 208/416] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:22.565740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:22.569125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.574903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:22.575521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:22.575795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:22.576085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] PATH_INFO: `/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:22.577208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b22621efa4e2c8d25ad2792cb5242)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:22.578659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:22.580591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 209/417] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:22 2022] GET /identity/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242 => generated 344 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:22.586915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.741059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180d60c-5752-4409-b795-87218ae0b654 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:22.767643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989aa50-6195-4859-8dab-95f809aa33a8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:22.780237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:22.781077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:22.781209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:22.781551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:22.782215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] RBAC: Adding query filter params (name=member) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:22.782514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:22.784018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:22.792965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 209/418] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:22.798674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.838274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:22.838944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:22.839215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:22.839484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] PATH_INFO: `/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242/groups/e12f441a1bcc4b0b842550c958fe0a3a/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:22.841482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6b22621efa4e2c8d25ad2792cb5242, group_id=e12f441a1bcc4b0b842550c958fe0a3a, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:22.843016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:22.857922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 210/419] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Oct 4 08:38:22 2022] PUT /identity/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242/groups/e12f441a1bcc4b0b842550c958fe0a3a/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:22.883079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:22.975710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227f14c-aa17-4e13-9c09-8910a0d2b16d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:23.037138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85aa55-f08e-4f68-9942-b33e430dd9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:23.037785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85aa55-f08e-4f68-9942-b33e430dd9ba None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:23.038061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85aa55-f08e-4f68-9942-b33e430dd9ba None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:23.038331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85aa55-f08e-4f68-9942-b33e430dd9ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:23.124554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85aa55-f08e-4f68-9942-b33e430dd9ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:23.126091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85aa55-f08e-4f68-9942-b33e430dd9ba None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:23.169685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:23.231907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 210/420] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:23.237304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:23.303294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:23.354241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad2932-f4e7-49ca-8ffa-9b1aa8d6edef None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:23.380995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64776-8e76-4ed6-952d-28f73f4eab80 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:23.381392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64776-8e76-4ed6-952d-28f73f4eab80 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:23.381586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64776-8e76-4ed6-952d-28f73f4eab80 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:23.382913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 211/421] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:38:23.387979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd11c56-7f5c-48dc-be6c-1b85f39ed4dd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:23.388263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd11c56-7f5c-48dc-be6c-1b85f39ed4dd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:23.388533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd11c56-7f5c-48dc-be6c-1b85f39ed4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:23.401964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:23.403026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:23.403283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:23.403555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:23.404753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:23.406384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:23.408410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 211/422] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:23 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:23.413858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:23.440894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2bd11c56-7f5c-48dc-be6c-1b85f39ed4dd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:23.442425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2bd11c56-7f5c-48dc-be6c-1b85f39ed4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:23.520953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa8c09-9998-4a46-aba3-ce884334ba17 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:23.552305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:23.560493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:23.560918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:23.561191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:23.561461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:23.562897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:23.565460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:23.570584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 212/423] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:23 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:23.571629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:23.587929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 212/424] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:38:23.717010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cd609-e61f-4a6b-83f2-5bb9c411d028 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:23.750769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:23.751412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:23.751801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:23.752095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:23.752814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:23.753113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:23.754528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:23.761039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 213/425] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:23 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:23.951924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd11c56-7f5c-48dc-be6c-1b85f39ed4dd None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:24.046910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:24.079446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd11c56-7f5c-48dc-be6c-1b85f39ed4dd None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:24.115966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a942c2-caca-4af0-a5e4-37565be0670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:24.116595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a942c2-caca-4af0-a5e4-37565be0670c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.116864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a942c2-caca-4af0-a5e4-37565be0670c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.117125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a942c2-caca-4af0-a5e4-37565be0670c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.127819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a942c2-caca-4af0-a5e4-37565be0670c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-72a942c2-caca-4af0-a5e4-37565be0670c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:24.132864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:24.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:24.134573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 213/426] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:24.139720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:24.212868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:24.258818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600cd08-cef9-46e7-830f-4d99fa6e8557 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:24.288520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d696f65-d989-4145-aa8f-a6d7fb51aa80 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.288837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d696f65-d989-4145-aa8f-a6d7fb51aa80 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.289062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d696f65-d989-4145-aa8f-a6d7fb51aa80 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.290789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 214/427] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:24.296825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:24.297438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.297711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.297977 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.298641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:24.298939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:24.301809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:24.303440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df14a-2597-4ff7-83b4-827b514c4f12 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.303723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df14a-2597-4ff7-83b4-827b514c4f12 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.304024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df14a-2597-4ff7-83b4-827b514c4f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.310856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 214/428] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:24.316513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:24.331200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-780df14a-2597-4ff7-83b4-827b514c4f12 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:24.334691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-780df14a-2597-4ff7-83b4-827b514c4f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:24.437279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 215/429] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:24.445826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b15f-e7f7-431c-8740-fb789e20c29f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:24.471383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:24.485297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:24.485921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.486200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.486469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.529360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f260-bffe-45eb-a46d-f3083ba657fe None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.529717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f260-bffe-45eb-a46d-f3083ba657fe None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.530003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f260-bffe-45eb-a46d-f3083ba657fe None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.531460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 216/430] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:24.537452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fbb56-ede7-49bd-afd0-a26a3d2fcba4 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.537997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fbb56-ede7-49bd-afd0-a26a3d2fcba4 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.538478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fbb56-ede7-49bd-afd0-a26a3d2fcba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.558810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-787fbb56-ede7-49bd-afd0-a26a3d2fcba4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:24.560662 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-787fbb56-ede7-49bd-afd0-a26a3d2fcba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:24.577877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:24.579350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:24.633989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:24.646222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 217/431] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:38:24.697532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 215/432] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:24.702989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fbb56-ede7-49bd-afd0-a26a3d2fcba4 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:24.752166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:24.783803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:24.807954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fbb56-ede7-49bd-afd0-a26a3d2fcba4 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:24.842846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:24.843479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.843756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.844051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] PATH_INFO: `/v3/projects/ec27f0f03e8249ff8c379f6356fc3491` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.845177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] RBAC: Authorizing `identity:get_project(project_id=ec27f0f03e8249ff8c379f6356fc3491)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:24.846697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:24.848566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 218/433] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:24 2022] GET /identity/v3/projects/ec27f0f03e8249ff8c379f6356fc3491 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:24.857119 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:24.874646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe039b-1d73-42d5-9373-1b4aed7ad4b7 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:24.909643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:24.910373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:24.910649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:24.910954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:24.921467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:24.926298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:24.927226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:24.928181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:24.932415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 216/434] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:24.933448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:24.933555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:24.990270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bf874-7076-46c2-a426-3be4b60d016a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.022749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.025726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.026356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.026634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.026905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.028033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.029458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.031292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 219/435] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:24 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:25.031815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.042882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a01037-b719-40ef-a586-59ee9ffb2808 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.076905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.077577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.077860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.078130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.078804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:25.079106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.080555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.093797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 217/436] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:25.093797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.140533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51af65-3aec-4b3d-af84-1c36464c9cd4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.167179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.176577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.177248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.177530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.177799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.188320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:25.192290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1d2ce-e07c-4d33-89b9-25b6fa7f0946 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:25.193516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:25.194415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:25.195239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 220/437] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:25.195939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.231824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.232457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.232735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.233005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.233730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=ec27f0f03e8249ff8c379f6356fc3491) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:25.234029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.235457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.241153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 218/438] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=ec27f0f03e8249ff8c379f6356fc3491&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:25.241923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.254167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.324491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f791f6-3213-4ad6-a6bd-0bbac7da9959 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.365100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec9607-f647-46de-91c7-70c3de5a96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.366045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec9607-f647-46de-91c7-70c3de5a96c0 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.366422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec9607-f647-46de-91c7-70c3de5a96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.366791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec9607-f647-46de-91c7-70c3de5a96c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.392316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0c01f-1eba-47b3-b0fc-53a7de4a5f33 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.436062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.440953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.441754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.442219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.442492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.443223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:25.443528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.445014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.451112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec9607-f647-46de-91c7-70c3de5a96c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.453377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec9607-f647-46de-91c7-70c3de5a96c0 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.459451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 219/439] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:25.465988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.543170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.576679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 221/440] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:25.578324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6cc43-8775-4bc2-b817-1cd7f4defce1 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.578781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6cc43-8775-4bc2-b817-1cd7f4defce1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.579220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6cc43-8775-4bc2-b817-1cd7f4defce1 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.581077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 222/441] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:25.581971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6cc43-8775-4bc2-b817-1cd7f4defce1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.583502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82899e-5905-4160-bde3-a0ebd56a4fd6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.619332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.619530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.619758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.620056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.683813 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.691274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.692772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.696939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6cc43-8775-4bc2-b817-1cd7f4defce1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.729608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5df12-bf0e-4ebe-b4dc-bafcef467809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.730234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df12-bf0e-4ebe-b4dc-bafcef467809 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.730511 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df12-bf0e-4ebe-b4dc-bafcef467809 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.730781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df12-bf0e-4ebe-b4dc-bafcef467809 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.731877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5df12-bf0e-4ebe-b4dc-bafcef467809 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.733322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5df12-bf0e-4ebe-b4dc-bafcef467809 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.735102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 223/442] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:25.736038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a3a3a-6608-48b5-8dd3-083fcaf60f0c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.736317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a3a3a-6608-48b5-8dd3-083fcaf60f0c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.736586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a3a3a-6608-48b5-8dd3-083fcaf60f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.753945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3f7a3a3a-6608-48b5-8dd3-083fcaf60f0c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:25.755126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3f7a3a3a-6608-48b5-8dd3-083fcaf60f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:25.758884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.790865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 220/443] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:25.796265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.821937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 224/444] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:25.822982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a3a3a-6608-48b5-8dd3-083fcaf60f0c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.891635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9d1c5-2940-4cdb-bc37-65f834518903 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.917468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.927253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.927876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.928358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.928640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.931318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.932791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.934617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 221/445] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:25.937790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a3a3a-6608-48b5-8dd3-083fcaf60f0c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:25.940392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:25.981875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:25.982509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:25.982789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:25.983057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] PATH_INFO: `/v3/projects/17313a63ac2f4b9e971725e2dd1f040a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:25.984460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] RBAC: Authorizing `identity:get_project(project_id=17313a63ac2f4b9e971725e2dd1f040a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:25.984907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:25.985962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:25.992400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 225/446] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/projects/17313a63ac2f4b9e971725e2dd1f040a => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:25.993215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:26.044282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52077e0-8ba1-4e3d-91ed-08d55c0f13b4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:26.076622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:26.077225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:26.077502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:26.077771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:26.078538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:26.078799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:26.080257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:26.086560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 222/447] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:26.096129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:26.105051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba070ad-459b-4a30-a9c3-40ebdd761f79 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:26.142708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:26.143305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:26.143553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:26.143837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] PATH_INFO: `/v3/groups/8e555b8a93054e6d937c5c1cde270ab9` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:26.145036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] RBAC: Authorizing `identity:get_group(group_id=8e555b8a93054e6d937c5c1cde270ab9)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:26.146534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:26.148225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 226/448] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:25 2022] GET /identity/v3/groups/8e555b8a93054e6d937c5c1cde270ab9 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:26.154573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:26.181139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:26.221444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d37a7-9243-42da-90c6-e6b137822db0 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:26.225064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:26.265467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:26.266126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:26.266401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:26.267482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:26.273816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4b14-22d0-4e31-9139-74e397faf112 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:26.279737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:26.284338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:26.285307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:26.286109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 223/449] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:26.291522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:26.311466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:26.312135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:26.312412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:26.312666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:26.313386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] RBAC: Adding query filter params (group.id=8e555b8a93054e6d937c5c1cde270ab9, role.id=503f8b8f2dc44b52a8f6901954713e7d, scope.project.id=17313a63ac2f4b9e971725e2dd1f040a) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:26.313694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:26.315145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:26.322323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 227/450] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:26 2022] GET /identity/v3/role_assignments?group.id=8e555b8a93054e6d937c5c1cde270ab9&scope.project.id=17313a63ac2f4b9e971725e2dd1f040a&role.id=503f8b8f2dc44b52a8f6901954713e7d => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:26.400685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df503f4-4d6b-4acd-9103-66df59ca38b1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:26.439921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:26.440557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:26.440829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:26.441092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:26.441756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:26.442051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:26.443513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:26.453038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 224/451] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:26.459111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:26.561340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:26.577333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768f565-8581-469b-b19a-bbabc93a0be9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:26.613126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:26.613739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:26.614017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:26.614288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:26.682239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:26.683617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:26.766096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:26.782445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 228/452] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:26.788336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:26.815603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:26.890815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d68d8b-4d96-4f2d-b9e9-2cfd5457e51e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:26.933864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:26.934407 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:26.934676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:26.934943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] PATH_INFO: `/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:26.936163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6b22621efa4e2c8d25ad2792cb5242)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:26.937608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:26.939199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 225/453] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:38:26 2022] GET /identity/v3/projects/5d6b22621efa4e2c8d25ad2792cb5242 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:26.945429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:27.048019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4037c-1bb8-44f4-8bfd-5cf194fbc676 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:27.083485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9121d01-25c0-4066-ac5b-db69f5815cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:27.084440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9121d01-25c0-4066-ac5b-db69f5815cef None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:27.084814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9121d01-25c0-4066-ac5b-db69f5815cef None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:27.085162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9121d01-25c0-4066-ac5b-db69f5815cef None admin] PATH_INFO: `/v3/groups/e12f441a1bcc4b0b842550c958fe0a3a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:27.087094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9121d01-25c0-4066-ac5b-db69f5815cef None admin] RBAC: Authorizing `identity:get_group(group_id=e12f441a1bcc4b0b842550c958fe0a3a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:27.089434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9121d01-25c0-4066-ac5b-db69f5815cef None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:27.091800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 229/454] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Oct 4 08:38:26 2022] GET /identity/v3/groups/e12f441a1bcc4b0b842550c958fe0a3a => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:27.098451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:27.195800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:27.204788 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325dd0-0a4b-4c8f-9739-e7a93702f887 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:27.238650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042812e1-152b-4b92-b322-30ba5597b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:27.239229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042812e1-152b-4b92-b322-30ba5597b44d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:27.239829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042812e1-152b-4b92-b322-30ba5597b44d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:27.240136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042812e1-152b-4b92-b322-30ba5597b44d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:27.240859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042812e1-152b-4b92-b322-30ba5597b44d None admin] RBAC: Adding query filter params (group.id=e12f441a1bcc4b0b842550c958fe0a3a, role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=5d6b22621efa4e2c8d25ad2792cb5242) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:27.241161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042812e1-152b-4b92-b322-30ba5597b44d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:27.242568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042812e1-152b-4b92-b322-30ba5597b44d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:27.250509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 226/455] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Oct 4 08:38:27 2022] GET /identity/v3/role_assignments?group.id=e12f441a1bcc4b0b842550c958fe0a3a&scope.project.id=5d6b22621efa4e2c8d25ad2792cb5242&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:29.745287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efb2e-3205-4ab1-a15b-f56bfdbdf034 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:29.746131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efb2e-3205-4ab1-a15b-f56bfdbdf034 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:29.746211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efb2e-3205-4ab1-a15b-f56bfdbdf034 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:29.747996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 230/456] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:29.753103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798736f-27cf-4a22-9bb6-77f40421d7fa None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:29.753486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798736f-27cf-4a22-9bb6-77f40421d7fa None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:29.753653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798736f-27cf-4a22-9bb6-77f40421d7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:29.776829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a798736f-27cf-4a22-9bb6-77f40421d7fa None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:29.778072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a798736f-27cf-4a22-9bb6-77f40421d7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:29.860699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 227/457] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:30.199803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da152a0c-0b6b-4200-842d-17a5d9fc417b None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.200516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da152a0c-0b6b-4200-842d-17a5d9fc417b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.200991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da152a0c-0b6b-4200-842d-17a5d9fc417b None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.202833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 231/458] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:30.208609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0160240-9bfc-43e1-b491-f1b5c620b405 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.209118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0160240-9bfc-43e1-b491-f1b5c620b405 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.209586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0160240-9bfc-43e1-b491-f1b5c620b405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.227697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da152a0c-0b6b-4200-842d-17a5d9fc417b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:30.244820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d0160240-9bfc-43e1-b491-f1b5c620b405 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:30.246308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d0160240-9bfc-43e1-b491-f1b5c620b405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:30.331490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:30.355519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 228/459] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:38:30.357236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285026d4-776c-467b-83a6-27a32a59752c None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.357827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285026d4-776c-467b-83a6-27a32a59752c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.358328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285026d4-776c-467b-83a6-27a32a59752c None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.360329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 229/460] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:30.367086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fcec0-391e-43fa-9a2f-227b17e9a6c6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.367761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fcec0-391e-43fa-9a2f-227b17e9a6c6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.368259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fcec0-391e-43fa-9a2f-227b17e9a6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.387310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da152a0c-0b6b-4200-842d-17a5d9fc417b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:30.389038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a70fcec0-391e-43fa-9a2f-227b17e9a6c6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:30.390640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a70fcec0-391e-43fa-9a2f-227b17e9a6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:30.404284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:30.433486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c264ac1-acf8-4ced-acb5-e11dd3d0ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:30.434126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c264ac1-acf8-4ced-acb5-e11dd3d0ebf1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.434396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c264ac1-acf8-4ced-acb5-e11dd3d0ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.434658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c264ac1-acf8-4ced-acb5-e11dd3d0ebf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.454290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c264ac1-acf8-4ced-acb5-e11dd3d0ebf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:30.455684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c264ac1-acf8-4ced-acb5-e11dd3d0ebf1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-7c264ac1-acf8-4ced-acb5-e11dd3d0ebf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:30.488904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:30.489970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:30.490856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:30.490856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:30.490856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:30.491245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 232/461] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:30.496006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa02d5c-829c-4975-9aad-5ace72d78db1 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.496006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa02d5c-829c-4975-9aad-5ace72d78db1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.496468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa02d5c-829c-4975-9aad-5ace72d78db1 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.499550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 233/462] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:38:30.500293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa02d5c-829c-4975-9aad-5ace72d78db1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:30.520665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 230/463] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:30.522733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560208b2-70d8-4677-88b3-07ca4ed7f377 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.523212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560208b2-70d8-4677-88b3-07ca4ed7f377 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.524051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560208b2-70d8-4677-88b3-07ca4ed7f377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.543665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-560208b2-70d8-4677-88b3-07ca4ed7f377 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:30.545437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-560208b2-70d8-4677-88b3-07ca4ed7f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:30.622564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 231/464] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:30.643214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560208b2-70d8-4677-88b3-07ca4ed7f377 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:30.679471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa02d5c-829c-4975-9aad-5ace72d78db1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:30.696891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:30.750137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:30.751583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.752133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.752597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.753428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:30.753919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:30.756396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:30.777260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 234/465] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:30.782879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:30.803739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:30.836041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560208b2-70d8-4677-88b3-07ca4ed7f377 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:30.881970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:30.882869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.883148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.883420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.897878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:30.900028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:30.905161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b5ca-7b2f-458a-8c26-02b71af1f462 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:30.918872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:30.919971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:30.921173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:30.921173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:30.921173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:30.921173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 232/466] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:30.921871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:30.948997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:30.949752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:30.950128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:30.950553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:30.951282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:30.953051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:30.978045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 235/467] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:38:30 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:30.978873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:31.008062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:31.057247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb874f-24e8-45c3-966d-5a872fe730e9 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.088332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:31.097398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.098211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.098587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.098985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.106634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d640e-ddba-488c-b5fb-52e474fa8113 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.112044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.113701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:31.131339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:31.132563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:31.133618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:31.133618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:31.133618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:31.134710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 233/468] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:31.135684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:31.155042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.155923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.156515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.156928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.158026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:31.158472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.160007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.174948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 236/469] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:31.176213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:31.257712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad5d61-7404-4184-a87f-d3e45cc78a33 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.278300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:31.298101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ee775-7686-4508-a88b-69f663f9a1a5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.305195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.306499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.306946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.307358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.322706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.324214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.340224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.341191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:31.341447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:31.342361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:31.343225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:31.343225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:31.343225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:31.343225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 234/470] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:31.343915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:31.344451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.344927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.345813 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:31.346619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.349395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.365790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 237/471] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:31.367549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:31.377288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:31.457039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767f599-978d-427f-ac09-6b02a93cb685 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.491781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aea621-ed42-426b-a24e-a4cf8c057432 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.508308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.509771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.510550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.510996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.512089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.514135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.531779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.532488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.532822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.533156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.533871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:31.534270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.535746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.543562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 235/472] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:38:31 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:31.544726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:31.559265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 238/473] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:31.565124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:31.571315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:31.653209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:31.672992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6d247-bbd3-44d2-a33a-7639ea8f0e14 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.711285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5588cf2-275d-4566-a113-8bfd129ed05b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:31.721490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9291-64c6-4de6-825a-f831862d49d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.722922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9291-64c6-4de6-825a-f831862d49d9 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.723292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9291-64c6-4de6-825a-f831862d49d9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.723635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9291-64c6-4de6-825a-f831862d49d9 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.724468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9291-64c6-4de6-825a-f831862d49d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.726669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9291-64c6-4de6-825a-f831862d49d9 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.758582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 236/474] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:38:31 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:31.759655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e435f3-1d50-492e-a3b5-a59662300eb4 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.760026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e435f3-1d50-492e-a3b5-a59662300eb4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.760305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e435f3-1d50-492e-a3b5-a59662300eb4 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.761559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 237/475] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:31.767614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:31.768395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.768677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.768946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.769378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194b5f-f987-417b-aae3-a3ceaa2a89c2 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.769617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:31.769789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194b5f-f987-417b-aae3-a3ceaa2a89c2 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.770025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87194b5f-f987-417b-aae3-a3ceaa2a89c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.771284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:31.793391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-87194b5f-f987-417b-aae3-a3ceaa2a89c2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:31.794598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-87194b5f-f987-417b-aae3-a3ceaa2a89c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:31.800607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 239/476] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:38:31 2022] POST /identity/v3/users => generated 249 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:38:31.875931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 238/477] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:32.168903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:32.231483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:32.278308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda07813-09cf-419c-bcda-72720334b300 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:32.319058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:32.319943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:32.320362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:32.320772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:32.321585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:32.323095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:32.332963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.sql.core [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 08:38:32.332963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 08:38:32.332963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [parameters: {'id': '6b237bfeaaab4f599dc22b9b856d8f5c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 08:38:32.332963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103427) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result.read() Oct 04 08:38:32.353386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:38:32.354318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application yield resource Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise self.value Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application session.commit() Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 08:38:32.355253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.session.flush() Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application compat.raise_( Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 08:38:32.356177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:38:32.357069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result.read() Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:38:32.357920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [parameters: {'id': '6b237bfeaaab4f599dc22b9b856d8f5c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:32.358708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 08:38:32.359596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:32.362050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 240/478] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Oct 4 08:38:32 2022] POST /identity/v3/roles => generated 144 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:32.368478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87194b5f-f987-417b-aae3-a3ceaa2a89c2 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:32.476459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:32.507680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87194b5f-f987-417b-aae3-a3ceaa2a89c2 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:32.549287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5e074-f47b-4fa0-b110-42f10e9280a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:32.549921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e074-f47b-4fa0-b110-42f10e9280a0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:32.550194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e074-f47b-4fa0-b110-42f10e9280a0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:32.550506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e074-f47b-4fa0-b110-42f10e9280a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:32.567926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5e074-f47b-4fa0-b110-42f10e9280a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-b7e5e074-f47b-4fa0-b110-42f10e9280a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:32.572699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:32.573587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:32.574430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 239/479] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:38:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:32.579695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:32.708992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bbe90-3d84-4768-a55c-09a34e05093e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:32.732161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:32.747942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65890a41-6b1f-4696-915f-15f4058d5687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:32.749104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65890a41-6b1f-4696-915f-15f4058d5687 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:32.749385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65890a41-6b1f-4696-915f-15f4058d5687 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:32.749657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65890a41-6b1f-4696-915f-15f4058d5687 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:32.750335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65890a41-6b1f-4696-915f-15f4058d5687 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:32.750639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65890a41-6b1f-4696-915f-15f4058d5687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:32.752025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65890a41-6b1f-4696-915f-15f4058d5687 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:32.760592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 241/480] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:33.438615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d827b37-9c99-49da-9ea6-50201cf0ca0d None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:33.439124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d827b37-9c99-49da-9ea6-50201cf0ca0d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:33.439200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d827b37-9c99-49da-9ea6-50201cf0ca0d None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:33.440776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 240/481] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:33.446195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040911-c300-4fed-9058-7657dbaf7890 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:33.446606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040911-c300-4fed-9058-7657dbaf7890 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:33.446802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040911-c300-4fed-9058-7657dbaf7890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:33.472874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-25040911-c300-4fed-9058-7657dbaf7890 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:33.474224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-25040911-c300-4fed-9058-7657dbaf7890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:33.563660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 242/482] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:33.931367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d827b37-9c99-49da-9ea6-50201cf0ca0d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:34.075448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d827b37-9c99-49da-9ea6-50201cf0ca0d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:34.122700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7be7f4-d8b9-44c0-a3a7-bb9a53269ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:34.123342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7be7f4-d8b9-44c0-a3a7-bb9a53269ccb None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.123615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7be7f4-d8b9-44c0-a3a7-bb9a53269ccb None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.123905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7be7f4-d8b9-44c0-a3a7-bb9a53269ccb None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.125889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:34.135374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7be7f4-d8b9-44c0-a3a7-bb9a53269ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-db7be7f4-d8b9-44c0-a3a7-bb9a53269ccb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:34.140152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:34.141036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:34.141874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 241/483] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:34.148995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040911-c300-4fed-9058-7657dbaf7890 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:34.292092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040911-c300-4fed-9058-7657dbaf7890 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:34.297402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef275a3d-ce2b-46d7-9c4c-4961efc953d4 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.297687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef275a3d-ce2b-46d7-9c4c-4961efc953d4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.297958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef275a3d-ce2b-46d7-9c4c-4961efc953d4 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.304932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 242/484] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:34.311319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:34.316219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d282e-bae2-4cab-ab5b-0002eac083c9 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.316503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d282e-bae2-4cab-ab5b-0002eac083c9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.316775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d282e-bae2-4cab-ab5b-0002eac083c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.344701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:34.345318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.345591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.345854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.346528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:34.346825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:34.348234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:34.350688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-902d282e-bae2-4cab-ab5b-0002eac083c9 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:34.351936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-902d282e-bae2-4cab-ab5b-0002eac083c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:34.367008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 243/485] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:34.373565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:34.488810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 243/486] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:38:34.529895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b4a13-1adf-4718-a60b-a8c17c8c96e5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:34.565404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f31e-cfd9-4560-8e2f-2f85b3d44fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:34.565962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f31e-cfd9-4560-8e2f-2f85b3d44fdf None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.566239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f31e-cfd9-4560-8e2f-2f85b3d44fdf None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.566510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f31e-cfd9-4560-8e2f-2f85b3d44fdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.577754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f31e-cfd9-4560-8e2f-2f85b3d44fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:34.579203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f31e-cfd9-4560-8e2f-2f85b3d44fdf None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-d0d1f31e-cfd9-4560-8e2f-2f85b3d44fdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:34.593645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:34.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:34.595431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:34.595431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:34.595431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:34.595431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 244/487] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:34.599747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf659b-31b9-4ca4-b6cb-30593c1cb683 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.600057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf659b-31b9-4ca4-b6cb-30593c1cb683 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.600330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf659b-31b9-4ca4-b6cb-30593c1cb683 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.601751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 244/488] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:34.603149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212110e-b186-4679-af17-6070fc2ab9b7 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.603431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212110e-b186-4679-af17-6070fc2ab9b7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.603698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212110e-b186-4679-af17-6070fc2ab9b7 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.605190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 245/489] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:34.607743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf659b-31b9-4ca4-b6cb-30593c1cb683 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:34.608083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37983-24f1-41fc-930c-7c15fca62ae0 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.608476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37983-24f1-41fc-930c-7c15fca62ae0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.608750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37983-24f1-41fc-930c-7c15fca62ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.630834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c8c37983-24f1-41fc-930c-7c15fca62ae0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:34.632043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c8c37983-24f1-41fc-930c-7c15fca62ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:34.661707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:34.728390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 246/490] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:34.729723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f87292-1abd-4a0d-a38b-2c153f8a912c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.730002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f87292-1abd-4a0d-a38b-2c153f8a912c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.730270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f87292-1abd-4a0d-a38b-2c153f8a912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.759809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf659b-31b9-4ca4-b6cb-30593c1cb683 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:34.766455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-77f87292-1abd-4a0d-a38b-2c153f8a912c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:34.767698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-77f87292-1abd-4a0d-a38b-2c153f8a912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:34.808615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:34.827403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:34.827954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:34.828229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:34.828794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:34.833523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:34.833840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:34.835232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:34.848034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 245/491] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:34.848999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:34.862948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 247/492] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:34.863844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f87292-1abd-4a0d-a38b-2c153f8a912c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:34.954826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:34.966985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f87292-1abd-4a0d-a38b-2c153f8a912c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.002777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.003406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.003742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.004079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.011374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.012838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:35.026831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:35.027981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:35.029151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:35.031336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 248/493] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:38:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:35.031692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c763ed-7960-46cb-b737-5ac50c30ce2b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.037689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.085658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.086211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.086487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.086944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.098220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:35.104242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:35.105352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:35.106157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:35.108955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 246/494] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:35.109889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.162063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:35.167414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477541f-1b7f-426f-b1ef-540e524c5cc8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.205099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.206050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.206500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.207014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.207900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:35.208375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.210189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.215162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe09b3-4ce3-43ea-8c00-4a34da250db9 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.249381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.252020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 249/495] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:35.252150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.252150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.252150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.252365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.260980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:35.262711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:35.267950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:35.268840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:35.269669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 247/496] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:35.270358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.369623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234771-4172-418f-bed5-6842cddeb6cd None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.369802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140924b-d802-43da-9860-57aaf34d62e3 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.372340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:35.406739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.407341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.407612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.407799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781900d2-d772-44eb-a292-b509d394245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.407934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.408457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781900d2-d772-44eb-a292-b509d394245c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.408566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:35.408768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781900d2-d772-44eb-a292-b509d394245c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.408871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.409016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781900d2-d772-44eb-a292-b509d394245c None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.410171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.419050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781900d2-d772-44eb-a292-b509d394245c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:35.423912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 250/497] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-781900d2-d772-44eb-a292-b509d394245c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:35.424027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:35.424879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:35.429159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:35.429797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.429872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 248/498] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:35.429872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781900d2-d772-44eb-a292-b509d394245c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.503300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:35.531505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781900d2-d772-44eb-a292-b509d394245c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.538966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8fbf3-673f-4722-91c2-4627c8735535 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.571360 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.572175 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.572547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.572827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.573781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:35.574095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.574836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.575451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.575599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.575758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.576029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.591708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 249/499] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:35.591708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.623760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:35.657627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.659193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.701046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617ed4e-6003-40aa-9ad3-a61e9ae38343 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.730689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:35.733723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.734357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.734638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.734914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.745312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.747073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:35.761087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:35.762130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:35.763206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:35.763206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:35.763206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:35.763206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 250/500] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:35.763206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.767123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 251/501] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:35.768042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.843978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:35.865597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2264fa3-76f7-4e17-8810-b6dea682b125 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.866183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5466d-f4e4-409f-91cc-512552ac7c32 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:35.898843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.899300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e3de5-6729-41e7-bbd6-8fe1ddfc0b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:35.899428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.899654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.899930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e3de5-6729-41e7-bbd6-8fe1ddfc0b23 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.900035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.900216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e3de5-6729-41e7-bbd6-8fe1ddfc0b23 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.900493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e3de5-6729-41e7-bbd6-8fe1ddfc0b23 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.901161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e3de5-6729-41e7-bbd6-8fe1ddfc0b23 None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:35.901463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e3de5-6729-41e7-bbd6-8fe1ddfc0b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.902851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e3de5-6729-41e7-bbd6-8fe1ddfc0b23 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.909611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:35.910826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 251/502] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:35.911053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:35.912070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d266-d3a3-4db9-bfc0-ee69b560701f None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:35.912353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d266-d3a3-4db9-bfc0-ee69b560701f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:35.912627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d266-d3a3-4db9-bfc0-ee69b560701f None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:35.913810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 252/503] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:35.915837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7d266-d3a3-4db9-bfc0-ee69b560701f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:35.929293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:35.930398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:35.931380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:35.931380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:35.931380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:35.934944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 252/504] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:35.935808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:35.957886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.022551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7d266-d3a3-4db9-bfc0-ee69b560701f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.041606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e43921-03d9-4457-9fbb-77ef46075852 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.059029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.059648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.059950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.060225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.060870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:36.061174 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.062547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.071356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.074872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 253/505] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:36.076123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.076258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5801c-614d-4ada-b069-5796fde75f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.076886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5801c-614d-4ada-b069-5796fde75f8b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.077173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5801c-614d-4ada-b069-5796fde75f8b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.077420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5801c-614d-4ada-b069-5796fde75f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.088219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5801c-614d-4ada-b069-5796fde75f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.089715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5801c-614d-4ada-b069-5796fde75f8b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-9ca5801c-614d-4ada-b069-5796fde75f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:36.104170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:36.105160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:36.106442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:36.106442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:36.106442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:36.106442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 253/506] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:36.106442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d5d66-6662-43e5-848c-8c52aa77e2c6 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.106783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d5d66-6662-43e5-848c-8c52aa77e2c6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.107011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d5d66-6662-43e5-848c-8c52aa77e2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.125842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d95d5d66-6662-43e5-848c-8c52aa77e2c6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:36.127046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d95d5d66-6662-43e5-848c-8c52aa77e2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:36.181666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ca136-3a9d-46dc-849d-d94165891d41 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.191194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 254/507] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:36.192046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d5d66-6662-43e5-848c-8c52aa77e2c6 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.194243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.215097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.215511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.215783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.216072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.227409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.228889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:36.243120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:36.244284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:36.245629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:36.245629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:36.245629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:36.245629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 254/508] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:36.245629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.297326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.307798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d5d66-6662-43e5-848c-8c52aa77e2c6 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.342787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.343398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.343657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.343974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.344614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:36.344917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.346289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.349377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b9aa0-6ff5-43ee-a25a-c969d50fdc14 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.357581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 255/509] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:36.358380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.391731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.392390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.392924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.392924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] PATH_INFO: `/v3/users/glance` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.400811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.402217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:36.414634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:36.415604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:36.417223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 255/510] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:36.420919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.432673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.473824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c510f-fc6b-4912-a35b-582fe5c49f60 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.506739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.507342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.507624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.507910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.508557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:36.508859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.510210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.522012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 256/511] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:36.522778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.526559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fddcebd-b60c-4949-8921-392d3e9d2108 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.539522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.561317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.561921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.562193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.562457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.563091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:36.563387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.564787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.576368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 256/512] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:36.577157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.625587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1cb84-91ff-402d-9fb1-684823d02c58 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.649714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.662874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.663500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.663774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.664073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.671093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.672570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.673872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3638142-4d2a-4d67-ba6f-40cd8949d75a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:36.683853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:36.684929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 257/513] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:36.686206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.713058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.713661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.713935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.714200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.714927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:36.715226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.716718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.754582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 257/514] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:36.755509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.786529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.798773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e3894-2c07-4817-9646-b6be1aaedc2e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.832544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.833242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.833477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.833742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.844868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.846314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.854418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39a9c0-741e-4aa7-a97f-ba558ec859ad None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:36.860137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:36.861198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:36.862151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:36.862151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:36.862151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:36.862151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 258/515] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:36.862480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.889461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:36.890100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:36.890362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:36.890628 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:36.895385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:36.897473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:36.899024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:36.910824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:36.911878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:36.912794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 258/516] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:36.913272 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:36.972158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f5b36-1570-4dd2-928a-e281318cb99e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.000434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.007421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.008077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.008354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.008652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.010414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d63fc-60da-4755-b8ea-c5b6ce168177 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.015981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.017412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:37.028584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:37.029561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:37.030459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 259/517] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:37.031019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.046058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.046667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.046943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.047213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.047981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:37.048287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.049739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.066491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 259/518] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Oct 4 08:38:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:37.067330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.124114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.130674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c180b-2647-4faa-bda9-bfa8eebf0529 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.163623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.164257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.164531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.164639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5d5dd-11f5-420e-ba83-5c1ca8943225 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.164792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.196776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.197364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.197641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.197908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.198536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:37.198838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.200229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.211082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 260/519] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:37.211846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.222024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.234211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.235751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.307007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938339-a5ae-4664-a15c-8acd823ee89e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.322833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.330531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 260/520] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:37.331327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.339838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.340468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.340744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.341046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.341712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:37.342013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.343443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.361541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 261/521] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:37.362320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.427588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.429742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36be29c-aa13-4914-bf9a-70dc8bb6ba4f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.462284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d888ae6-ded0-4e74-96e8-6ddf7d3109c2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.462553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.463155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.463433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.463676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.464454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:37.464752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.466163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.482708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 261/522] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:37.483412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.496309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.496912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.497187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.497456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.538360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.564249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.565726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.585155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a363b1-c2b4-4048-a4d9-631053478fb0 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.617906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.618510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.618783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.619087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.619712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.621495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.628863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.642018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 262/523] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:38:37 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:37.642817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.681631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 262/524] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:37.681631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.763593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7cfee-3fd0-4d55-a741-9662e47e3dbb None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.774730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.790752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32913f-c594-4cfc-8e83-5f12d7c5404a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.807648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.808304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.808646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.808920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.822831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.824334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.826919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:37.827555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.827827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.828198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.828960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=bf149f246c1644aa85c4c162a2e07984) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:37.829676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:37.830684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:37.837321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 263/525] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/role_assignments?user.id=bf149f246c1644aa85c4c162a2e07984&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:37.838025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:37.841200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:37.842114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:37.842975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:37.842975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:37.842975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:37.842975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 263/526] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:37.843209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.879260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.968988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483a09a-f751-4316-b88c-9e975502ba01 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:37.971664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a2d35-6176-498c-aa6e-ad2057deb233 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.009015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.009822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.010478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.010478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.015911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.016732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.017170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.017576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.024583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.026231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:38.042447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:38.043542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:38.044780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:38.044780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:38.044780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:38.044780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 264/527] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:38.045905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.062285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.109049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.111284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.150090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.165139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d4c2-bc94-4536-ae2f-11829ccf2695 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.201872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.202480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.202758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.203019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.203661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:38.203983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.205934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.219708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 265/528] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:38.220806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.225526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 264/529] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:38.226435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.296235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.329776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d11e7-8ef6-42d7-974d-cb5a003fe5db None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.332966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b648e-6c39-4d31-bc4a-c14acf35f774 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.365692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.370388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.371371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.371645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.371940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.372581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:38.372876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.373860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.374222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.374830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.375257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.375663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.382980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.384660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.387472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 266/530] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:38.388281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:38.397266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:38.398325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:38.399352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:38.401261 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 265/531] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:38.402162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.502059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109a01a-0df8-4f50-9668-5ce51c718761 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.505685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab725e-7797-42aa-9e85-3f1c613d8750 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.533982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.540184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.540800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.541079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.541339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.542148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.542991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.543404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.543822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.550994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.552349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.553017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.553755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:38.565680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:38.567092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:38.568170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:38.569087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:38.569976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:38.570877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:38.570877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:38.570877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:38.570877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 267/532] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:38.571314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.572425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 266/533] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:38.576783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.636314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.685697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb2bd5-e7d8-4cff-ac6e-029b9a3e5bad None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.695637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601b557-c0fe-4f20-92f9-1e1865295116 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.724176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.725175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.725580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.725859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.726561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:38.726849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.728302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.733067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.733916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.734338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.734787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.735615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:38.736135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.737655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.748380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 268/534] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:38.749543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.751196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 267/535] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:38.753678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.809086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.864967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff6983-552c-41af-87e1-307020ccc80a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.889522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfee1d-bffe-48c4-ae5b-7f27ac21b405 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.902626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.911958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.913037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.913693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.914105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.915553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:38.916029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.918573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.925034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.925634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.925907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.926231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.926985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=557974bcc89c412f8d5c2e2039843f65) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:38.927291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.928761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.934734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 269/537] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:38.934863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 268/537] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/role_assignments?user.id=557974bcc89c412f8d5c2e2039843f65&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:38.941523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:38.941660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:39.095431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e47a3-b04a-4c13-aa74-4e26d4872c43 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:39.151547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf699ad-9ec5-4b5f-9c09-52dc954e0cb7 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:39.154497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:39.172704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:39.173405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:39.173621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:39.173885 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:39.186825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:39.192395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:39.193249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:39.193668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:39.194080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:39.194965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=349ee49c297749468e09c4aa3dc9c218) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:39.195415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:39.196498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:39.197396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:39.203991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 270/538] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/role_assignments?user.id=349ee49c297749468e09c4aa3dc9c218&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:39.213601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:39.214656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:39.215707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 269/539] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:39.220940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:39.358112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b08e5-447f-4992-96fd-f09519ce0284 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:39.407150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92579150-2749-460b-921e-8880cf057eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:39.407763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92579150-2749-460b-921e-8880cf057eb4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:39.408064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92579150-2749-460b-921e-8880cf057eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:39.408361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92579150-2749-460b-921e-8880cf057eb4 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:39.409057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92579150-2749-460b-921e-8880cf057eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:39.409360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92579150-2749-460b-921e-8880cf057eb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:39.410779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92579150-2749-460b-921e-8880cf057eb4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:39.423567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 271/540] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:39.429742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:39.465278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:39.531187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e3d0d-dca3-47e4-93e7-e2ed3058f8fe None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:39.565729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0974fd-2ce2-47b4-886d-776c62a5ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:39.566374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0974fd-2ce2-47b4-886d-776c62a5ad4b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:39.566631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0974fd-2ce2-47b4-886d-776c62a5ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:39.566855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0974fd-2ce2-47b4-886d-776c62a5ad4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:39.567651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0974fd-2ce2-47b4-886d-776c62a5ad4b None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=d877daa4c2c14d0c8d281aeebab27ee1) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:39.567983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0974fd-2ce2-47b4-886d-776c62a5ad4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:39.569421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0974fd-2ce2-47b4-886d-776c62a5ad4b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:39.580018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 270/541] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:39 2022] GET /identity/v3/role_assignments?user.id=d877daa4c2c14d0c8d281aeebab27ee1&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:40.080512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4367f9-d025-40a6-84d1-2fefeb905d98 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:40.081023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4367f9-d025-40a6-84d1-2fefeb905d98 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:40.081129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4367f9-d025-40a6-84d1-2fefeb905d98 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:40.082521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 272/542] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:40.088248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88987-1254-4b6e-814b-4e75fd697bcb None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:40.088530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88987-1254-4b6e-814b-4e75fd697bcb None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:40.088802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f88987-1254-4b6e-814b-4e75fd697bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:40.113023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-56f88987-1254-4b6e-814b-4e75fd697bcb None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:40.114225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-56f88987-1254-4b6e-814b-4e75fd697bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:40.202589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 271/543] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:40.466424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd6f6-9a06-4853-83cb-628207a40844 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:40.466800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd6f6-9a06-4853-83cb-628207a40844 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:40.467008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd6f6-9a06-4853-83cb-628207a40844 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:40.468595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 273/544] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:40.487062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b032ff6-92b6-441b-8ac7-8466949295dd None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:40.487338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b032ff6-92b6-441b-8ac7-8466949295dd None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:40.487612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b032ff6-92b6-441b-8ac7-8466949295dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:40.507756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2b032ff6-92b6-441b-8ac7-8466949295dd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:40.508957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2b032ff6-92b6-441b-8ac7-8466949295dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:40.538690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:40.561922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dd6f6-9a06-4853-83cb-628207a40844 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:40.597879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:40.603154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 272/545] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:40.689513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dd6f6-9a06-4853-83cb-628207a40844 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:40.736170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:40.737028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:40.737490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:40.737895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:40.754719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:40.759804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:40.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:40.761916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:40.763849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 274/546] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:40.769975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b032ff6-92b6-441b-8ac7-8466949295dd None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:40.894600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:40.919547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b032ff6-92b6-441b-8ac7-8466949295dd None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.000646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.001474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.001896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.002323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.003131 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:41.003586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.004476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.005429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.017951 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 273/547] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:41.019189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f8a69-60ec-41c9-ac5a-5fa3e10e2f68 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.023603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.054919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.055966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.056457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.056956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.068421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:41.073853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:41.074768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:41.075587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:41.078370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 275/548] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:41.084833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.092286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.203327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04798a8d-c32a-44a9-b2fd-6fbe547f90c8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.240526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.241145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.241427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.241703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.242370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:41.242679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.244074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.253913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 276/549] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:41.259087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.276258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.294027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80324bb8-edfc-4d51-a76b-f29d767dab45 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.357679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.358316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.358592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.358858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.380592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.382271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:41.415100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:41.416440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:41.417322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:41.417322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:41.417322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:41.417322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 274/550] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:41.424111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.447396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc703eb-8b3e-4cac-a3aa-f4d831fbdbb6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.496270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.496906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.497180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.497443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.515808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.517206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:41.542555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:41.544721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:41.546896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:41.546896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:41.546896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:41.553284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 277/551] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:41.565543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.583259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.611273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6ef41-6703-48a5-8151-a271907948aa None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.634523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.650021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edadefc-f3a2-462b-9294-f32df796933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.650774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edadefc-f3a2-462b-9294-f32df796933c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.651163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edadefc-f3a2-462b-9294-f32df796933c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.651526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edadefc-f3a2-462b-9294-f32df796933c None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.652386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edadefc-f3a2-462b-9294-f32df796933c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:41.652785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edadefc-f3a2-462b-9294-f32df796933c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.654370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edadefc-f3a2-462b-9294-f32df796933c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.668620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 275/552] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:41.670249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc21fc4-70bd-4285-95b8-54a28b45017e None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.670635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc21fc4-70bd-4285-95b8-54a28b45017e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.670991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc21fc4-70bd-4285-95b8-54a28b45017e None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.676136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 276/553] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:41.677227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc21fc4-70bd-4285-95b8-54a28b45017e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.687969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cf002-cb33-4f13-a85c-74544bd87e76 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.725056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932943d-95c6-4148-9084-a0ed1448df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.725682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932943d-95c6-4148-9084-a0ed1448df41 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.725961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932943d-95c6-4148-9084-a0ed1448df41 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.726230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932943d-95c6-4148-9084-a0ed1448df41 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.726883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932943d-95c6-4148-9084-a0ed1448df41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:41.727184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932943d-95c6-4148-9084-a0ed1448df41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.728600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932943d-95c6-4148-9084-a0ed1448df41 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.742147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 278/554] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:41.743450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df3429-a1bb-4706-9aef-09bdcff96d5c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.744670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df3429-a1bb-4706-9aef-09bdcff96d5c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.744945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df3429-a1bb-4706-9aef-09bdcff96d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.767314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-22df3429-a1bb-4706-9aef-09bdcff96d5c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:41.768672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-22df3429-a1bb-4706-9aef-09bdcff96d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:41.818065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc21fc4-70bd-4285-95b8-54a28b45017e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.852212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 279/555] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:41.853452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df3429-a1bb-4706-9aef-09bdcff96d5c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.868743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bd6d7-0afb-4576-a635-9f3eb4e3589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.869643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bd6d7-0afb-4576-a635-9f3eb4e3589d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.870177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bd6d7-0afb-4576-a635-9f3eb4e3589d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.871025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bd6d7-0afb-4576-a635-9f3eb4e3589d None admin] PATH_INFO: `/v3/users/swift` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.878521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.880747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bd6d7-0afb-4576-a635-9f3eb4e3589d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.882478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1bd6d7-0afb-4576-a635-9f3eb4e3589d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-aa1bd6d7-0afb-4576-a635-9f3eb4e3589d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:41.895544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:41.896606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:41.897537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 277/556] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:41.898758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbfcf-6795-4bbf-835d-3bb7702a855d None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.899107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbfcf-6795-4bbf-835d-3bb7702a855d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.899454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbfcf-6795-4bbf-835d-3bb7702a855d None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.904902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 278/557] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:41.905743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbfcf-6795-4bbf-835d-3bb7702a855d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.917455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.986976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df3429-a1bb-4706-9aef-09bdcff96d5c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.024267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d96ba-0ea3-4024-9171-3849078a57a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.025546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d96ba-0ea3-4024-9171-3849078a57a5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.025546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d96ba-0ea3-4024-9171-3849078a57a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.025840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d96ba-0ea3-4024-9171-3849078a57a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.034460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d96ba-0ea3-4024-9171-3849078a57a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.036084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d96ba-0ea3-4024-9171-3849078a57a5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.042727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbfcf-6795-4bbf-835d-3bb7702a855d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-6b4d96ba-0ea3-4024-9171-3849078a57a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:42.050204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:42.051921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:42.053008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:42.053560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 280/558] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:42.054944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2aeec-0be1-4e5c-90b8-cea628223fe9 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.055302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2aeec-0be1-4e5c-90b8-cea628223fe9 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.056007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2aeec-0be1-4e5c-90b8-cea628223fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.075475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b1f2aeec-0be1-4e5c-90b8-cea628223fe9 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:42.076892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b1f2aeec-0be1-4e5c-90b8-cea628223fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:42.092753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.093870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.094324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.094970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.096285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:42.096783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.098388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.123812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 279/559] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:42.124683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.156642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 281/560] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:42.157513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2aeec-0be1-4e5c-90b8-cea628223fe9 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.178606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.201066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.227151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e76db6-232a-4715-a4f2-2872a43b2295 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.264854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5540fd-ce06-4a5b-b90f-9188f620493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.265550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5540fd-ce06-4a5b-b90f-9188f620493e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.265765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5540fd-ce06-4a5b-b90f-9188f620493e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.266005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5540fd-ce06-4a5b-b90f-9188f620493e None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.266731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5540fd-ce06-4a5b-b90f-9188f620493e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:42.267032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5540fd-ce06-4a5b-b90f-9188f620493e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.267310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2aeec-0be1-4e5c-90b8-cea628223fe9 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.268502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5540fd-ce06-4a5b-b90f-9188f620493e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.287183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 280/561] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:42.288765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfde9f-3e1b-47d3-8a0c-d5bb1a4e26be None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.289706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfde9f-3e1b-47d3-8a0c-d5bb1a4e26be None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.290183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfde9f-3e1b-47d3-8a0c-d5bb1a4e26be None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.292179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 281/562] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:42.292884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfde9f-3e1b-47d3-8a0c-d5bb1a4e26be None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.307279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e9c56-3f91-45e6-821d-4b795e044e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.307899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e9c56-3f91-45e6-821d-4b795e044e8f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.308182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e9c56-3f91-45e6-821d-4b795e044e8f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.308446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e9c56-3f91-45e6-821d-4b795e044e8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.378020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e9c56-3f91-45e6-821d-4b795e044e8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.379532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e9c56-3f91-45e6-821d-4b795e044e8f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.398109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfde9f-3e1b-47d3-8a0c-d5bb1a4e26be None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.409561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.422458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.434097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.434722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.434997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.435267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.445263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:42.450322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:42.451311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 282/563] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:42.452265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.479106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 282/564] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:42.480757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441350c-20bf-4526-ac7e-5c994e264042 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.481054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441350c-20bf-4526-ac7e-5c994e264042 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.481312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441350c-20bf-4526-ac7e-5c994e264042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.498539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e441350c-20bf-4526-ac7e-5c994e264042 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:42.499701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e441350c-20bf-4526-ac7e-5c994e264042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:42.551122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c70094-6f3f-4a7f-b377-47f2b84c29b4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.566883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 283/565] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:42.567782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441350c-20bf-4526-ac7e-5c994e264042 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.602228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.602862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.603142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.603415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.616412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.618314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:42.633335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:42.634305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:42.635215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:42.635215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:42.635215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:42.635215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 283/566] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:42.635215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.642312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.666695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.676688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441350c-20bf-4526-ac7e-5c994e264042 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.714041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.714679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.714957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.715225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.725720 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:42.733080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:42.734424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 284/567] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:42.735263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.756755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4834911-c59d-424c-afa0-49b7435b1828 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.791642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.792343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.792616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.792879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.793538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:42.793835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.795212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.803183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 284/568] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:42.803917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.848170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36da186-924d-41e3-a4df-0dd5cd6829ea None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.879368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.884911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.885576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.885853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.886126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.887387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:42.896230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.897648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.902987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdccf88-e406-440a-a8ef-49985f0828be None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:42.911817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:42.912927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:42.913887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:42.913887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:42.913887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:42.913887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 285/569] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:42.914109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:42.941579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:42.942450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:42.943072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:42.943340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:42.944008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:42.944319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:42.945657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:42.958527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 285/570] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:42.959341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.016740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626a197-0e8f-4d4a-b86a-8726538dd840 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.054291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.054811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.055087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.055357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.056064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:43.056371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.057767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.064717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2a766-b525-4f6f-8f96-65b6e584fcab None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.065865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 286/571] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:43.066646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.099335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.099994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.100273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.100542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.110229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.111668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.114106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.115382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:43.127329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:43.128346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:43.129227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:43.129227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:43.129227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:43.129227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 286/572] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:43.130530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.170487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0219-5143-44fe-967a-d9f63d665e6b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.205038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.205679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.205948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.206210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.216040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:43.221104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:43.222117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 287/573] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:43.222958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.229029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a5f99-a3b0-42da-83b3-0f225a10ef4c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.262503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.263134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.263412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.263682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.264371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:43.264677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.266057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.277813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 287/574] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:43.278530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.328029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769795e-65f2-48a5-8130-dc29bf20b523 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.330997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.334899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.368743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.369378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.369654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.369912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.376847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.378278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.378403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498042d-88eb-4f4d-b1be-15f281e94561 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:43.389961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:43.390853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:43.391746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 288/575] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:43.392289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.413178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.413806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.414079 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.414347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.424365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.425823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:43.440139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:43.441083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:43.441978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:43.441978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:43.441978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:43.441978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 288/576] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:43.441978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.494674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb63fd-8074-41f8-88df-5acc88bd01cb None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.528190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.528785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.529056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.529321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.529945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:43.530237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.531576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.542911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 289/577] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:43.543670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.544159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d542b0-39ce-4a2c-b5ee-97cbbf5a66a5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.555452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.572545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.582747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.583539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.583906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.584246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.584991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:43.585355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.586795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.595052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 289/578] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:43.595840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.646939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c4d56-bd75-423f-a7a1-d16413bf790f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.680538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.681163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.681442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.681713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.688293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.689733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.691841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aba6f0-2aaa-43cb-b4de-dd29804cac4e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:43.701086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:43.702000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:43.702896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 290/579] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:43.703379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.728781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.729387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.729641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.729933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.730645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:43.730949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.732400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.744105 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 290/580] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:43.744867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.774415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.786591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:43.801412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1750b2-e42f-4d0b-8452-3f30efacd74b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.835337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.836183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.836450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.837088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:43.837390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.838740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.842460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9046ac4-b5b6-452f-862a-15b8565215c5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.850222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 291/581] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:43.850859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.879235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.879925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.880206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.880475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] PATH_INFO: `/v3/users/glance` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.887433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:43.888890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:43.900195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:43.901321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 291/582] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:43.902426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:43.952436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df79c6-07b4-46d4-9712-0779c9009ec4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:43.988450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:43.989175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:43.989544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:43.989818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:43.991300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.000210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.001603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bfda7-87e8-414a-abcd-5f2231b1f07a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.001717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.008517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:44.016646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:44.022777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:44.023906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:44.023906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:44.023906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:44.023906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 292/583] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:44.023906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.035319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.035947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.036253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.036527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.037214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:44.037518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.038946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.051208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 292/584] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:44.052201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.124928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540a14-9038-47ee-867c-1d1a8cd1624d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.151527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd496e-e620-40a9-a8aa-280a84378c21 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.156789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.157447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.157723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.157984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/bf149f246c1644aa85c4c162a2e07984/roles/4460b00f7dab40cd8eb26f0b606b4101` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.172632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=bf149f246c1644aa85c4c162a2e07984, role_id=4460b00f7dab40cd8eb26f0b606b4101)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.174060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.185314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.185943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.186223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.186495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.187206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 293/585] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:38:44 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/bf149f246c1644aa85c4c162a2e07984/roles/4460b00f7dab40cd8eb26f0b606b4101 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:44.188026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.202335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.204032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:44.217047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:44.218117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:44.219092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 293/586] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:44.219708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.233979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.257820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.322337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f8424-f3f1-4075-a792-42bb5bdf8ef4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.327439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d067d-5040-4c85-ba4c-95fd8053c4ac None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.365817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.366440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.366719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.366984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.368891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:44.368891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.369106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.369106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.369106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.369363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.369495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.369889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:44.370186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.373284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.386216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 294/587] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:44.386761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.390701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 294/588] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:44.391748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.456522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.490267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb10c2a-7dcd-4564-bb84-c610efc9a8f7 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.490819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f9b1e-c3de-4fb9-b723-b1ccca26e02d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.500146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.525085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.525856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.526118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.526384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.527457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:44.527722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.529170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.531354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.532092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.532463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.532825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.533643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:44.534040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.544297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 295/589] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/role_assignments?user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:44.544297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.544575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.570318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 295/590] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:44.571468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.663798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6643fd3-de2d-4c85-8cee-39ac73e8d244 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.690519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ab0f2-02ca-4005-8fe1-578ad8a4f0f4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.702595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01313d-9978-4887-9d41-331401348410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.703397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01313d-9978-4887-9d41-331401348410 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.703773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01313d-9978-4887-9d41-331401348410 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.704188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01313d-9978-4887-9d41-331401348410 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.712436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.714569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01313d-9978-4887-9d41-331401348410 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.716315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01313d-9978-4887-9d41-331401348410 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-2e01313d-9978-4887-9d41-331401348410 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:44.729853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:44.731073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:38:44.732311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:44.737450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.738271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.738497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 296/591] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:44.738829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.739327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.739795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01313d-9978-4887-9d41-331401348410 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.752130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.753756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:44.773059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:44.774104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:44.774994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:44.774994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:44.774994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:44.775166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 296/592] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:44.776900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.801172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.863295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01313d-9978-4887-9d41-331401348410 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.896999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0560b1-4818-4780-bb78-5975c4d427b2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:44.897998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.898729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.899107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.899477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.911037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.913035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:44.928519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:44.929672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:44.930763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:44.930763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:44.930763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:44.931862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 297/593] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:44.934641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.939537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:44.940530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.941001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.941480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.942572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:44.943109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:44.945212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:44.967437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 297/594] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:44.968686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.990621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.048830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2418b-3126-4ff0-a003-397ff9740a09 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.054296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.084243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cece7-96b9-4bbb-9aed-e98ebae0f656 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.084533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.084989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.085293 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.085536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.086230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:45.086581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.088709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.100115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 298/595] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:45.104963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.131460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.132431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.132848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.133272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.134140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:45.134670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.136263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.149998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 298/596] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:45.151091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.208298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1ae33-06f2-4032-816e-903b05b81aac None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.215617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.243382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.244052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.244336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.244606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.255266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.256746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.263032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e900d-86d4-4f05-8053-406e26f92c56 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:45.271828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:45.273012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:45.274072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:45.274072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:45.274072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:45.277716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 299/597] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:45.278578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.303589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.304588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.305006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.305490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.307712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.314410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.316173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:45.328392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:45.329546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:45.330534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:45.332458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 299/598] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:45.338083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.382048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87807d96-4ab3-4bbc-80e1-4f7fe4df9e5e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.416819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.417453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.417731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.418000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.426894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.428476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:45.444879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:45.446013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 300/599] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:45.446980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.459287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6565082-da3a-4f12-8fd4-3935c4810d21 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.469382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.498367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.499300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.499716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.500150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.500950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:45.501490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.503205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.516723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 300/600] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:45.517916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.558132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda8b9d-97f8-465b-b407-80c0bf3091bb None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.579034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.593045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.593667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.593932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.594192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.594885 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:45.595190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.596707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.613550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 301/601] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:45.614449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.629905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39c7ef-553e-4038-84b5-ad54088e12be None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.666507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.667330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.667747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.668203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.669126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:45.669681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.671225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.689471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 301/602] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:45.690605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.692086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.714960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a306e-7f37-4190-ac54-115f89dfad9d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.751107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.751754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.752063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.752334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.759816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.761700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:45.773684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:45.774766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:45.775680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 302/603] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:45.780286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.808081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778431cc-c067-4891-8ea8-31cfd3172d7b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.823457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.846704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.847675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.848224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.848726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/557974bcc89c412f8d5c2e2039843f65/roles/4460b00f7dab40cd8eb26f0b606b4101` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.864542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=557974bcc89c412f8d5c2e2039843f65, role_id=4460b00f7dab40cd8eb26f0b606b4101)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.866197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.882391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 302/604] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:38:45 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/557974bcc89c412f8d5c2e2039843f65/roles/4460b00f7dab40cd8eb26f0b606b4101 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:45.883470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:45.889588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492a4c-6c61-40e5-98aa-c7557edb3c6d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.958313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.958962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.959243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.959514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/349ee49c297749468e09c4aa3dc9c218/roles/4460b00f7dab40cd8eb26f0b606b4101` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.978755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:46.000901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=349ee49c297749468e09c4aa3dc9c218, role_id=4460b00f7dab40cd8eb26f0b606b4101)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:46.002423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:46.015002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efffe08-ae81-48ef-bfad-91b6f5d94f45 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:46.024191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 303/605] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:38:45 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/349ee49c297749468e09c4aa3dc9c218/roles/4460b00f7dab40cd8eb26f0b606b4101 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:46.065835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f06340-e81e-4711-8cec-12d033af99dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:46.066486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f06340-e81e-4711-8cec-12d033af99dc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:46.066815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f06340-e81e-4711-8cec-12d033af99dc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:46.067098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f06340-e81e-4711-8cec-12d033af99dc None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.068067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f06340-e81e-4711-8cec-12d033af99dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:46.068479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f06340-e81e-4711-8cec-12d033af99dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:46.085400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f06340-e81e-4711-8cec-12d033af99dc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:46.105953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 303/606] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:46.113077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:46.242257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abdad8-3e19-4187-b3ef-90c2c7a7473e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:46.281588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e2319-6c35-4281-b650-0b0aa569f69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:46.282254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e2319-6c35-4281-b650-0b0aa569f69d None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:46.282535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e2319-6c35-4281-b650-0b0aa569f69d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:46.282804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e2319-6c35-4281-b650-0b0aa569f69d None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/d877daa4c2c14d0c8d281aeebab27ee1/roles/4460b00f7dab40cd8eb26f0b606b4101` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.303585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e2319-6c35-4281-b650-0b0aa569f69d None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=d877daa4c2c14d0c8d281aeebab27ee1, role_id=4460b00f7dab40cd8eb26f0b606b4101)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:46.305095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e2319-6c35-4281-b650-0b0aa569f69d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:46.324207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 304/607] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:38:46 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/d877daa4c2c14d0c8d281aeebab27ee1/roles/4460b00f7dab40cd8eb26f0b606b4101 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:46.574020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53225a9-7a25-4455-82e3-a91c8f733c6d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:46.575119 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53225a9-7a25-4455-82e3-a91c8f733c6d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:46.575609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53225a9-7a25-4455-82e3-a91c8f733c6d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.577522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 304/608] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:46.583168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a5e01-159c-40d2-8127-9c26b8531135 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:46.583706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a5e01-159c-40d2-8127-9c26b8531135 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:46.584208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a5e01-159c-40d2-8127-9c26b8531135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.605334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-685a5e01-159c-40d2-8127-9c26b8531135 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:46.607005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-685a5e01-159c-40d2-8127-9c26b8531135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:46.637369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:46.680752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 305/609] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:46.931098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9e75-5872-4b1e-a30b-20893f5c9dbf None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:46.931510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9e75-5872-4b1e-a30b-20893f5c9dbf None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:46.931917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e9e75-5872-4b1e-a30b-20893f5c9dbf None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.933601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 305/610] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:46.938784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f40757-a112-40fa-991b-c477d474a936 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:46.939254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f40757-a112-40fa-991b-c477d474a936 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:46.939514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f40757-a112-40fa-991b-c477d474a936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.960314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-22f40757-a112-40fa-991b-c477d474a936 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:46.961623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-22f40757-a112-40fa-991b-c477d474a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:47.028080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9e75-5872-4b1e-a30b-20893f5c9dbf None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:47.053799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 306/611] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:47.063808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:47.175054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e9e75-5872-4b1e-a30b-20893f5c9dbf None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:47.214519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:47.215079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:47.215358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:47.215629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:47.229039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:47.233837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:47.234995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:47.236092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:47.236836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 306/612] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:47.240974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f40757-a112-40fa-991b-c477d474a936 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:47.385989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:47.389955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f40757-a112-40fa-991b-c477d474a936 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:47.435450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:47.436103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:47.436377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:47.436641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:47.437310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:47.437609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:47.438962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:47.456152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 307/613] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:47.461548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:47.480627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:47.636425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:47.656593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bced888-f5d9-4d9b-884e-6c1fec0ca708 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:47.669695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86695d-9a69-4db6-bcb7-997d52296c13 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:47.722487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:47.723130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:47.723402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:47.723726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:47.729274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:47.729912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:47.730186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:47.730457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:47.744708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:47.750220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:47.752397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:47.754157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 308/614] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:47.755831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:47.759907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:47.760116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:47.785187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:47.787513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:47.789704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:47.789704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:47.789704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:47.789704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 307/615] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:47.796037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:47.815731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:47.891910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb80e8-2377-4039-842e-2e570392c635 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:47.958864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b4224-d106-4fea-a1bc-251a7a30d4c6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:47.961442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:47.962122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:47.962398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:47.962715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:47.963383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:47.963680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:47.965083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:47.982805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 309/616] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:47.988837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.015971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.017453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.017731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.017999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.018644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:48.018943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.022834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.030700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:48.044064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 308/617] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:48.048548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.125256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:48.147046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd9e3f-a137-42a5-89ec-a8b28b2bd866 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.194867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.195526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.195804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.198141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.223394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.232432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.254605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6118e6-32e5-4837-96ea-ca83d770a99c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:48.257057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:48.259162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:48.261350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:48.261350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:48.261350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:48.261350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 310/618] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:48.267908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.297299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09540a3-5df5-4a72-896f-2bd79a51cbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.298084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09540a3-5df5-4a72-896f-2bd79a51cbb3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.298369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09540a3-5df5-4a72-896f-2bd79a51cbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.298645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09540a3-5df5-4a72-896f-2bd79a51cbb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.306854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09540a3-5df5-4a72-896f-2bd79a51cbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.308395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09540a3-5df5-4a72-896f-2bd79a51cbb3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c09540a3-5df5-4a72-896f-2bd79a51cbb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:48.321258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:48.322420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:48.323384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 309/619] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:38:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:48.324045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add818c-1af8-46bf-805b-790faf25ee2f None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.325725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add818c-1af8-46bf-805b-790faf25ee2f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.325725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add818c-1af8-46bf-805b-790faf25ee2f None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.325890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 310/620] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:48.330056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add818c-1af8-46bf-805b-790faf25ee2f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.391922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:48.406608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227b4dc-9c57-4cb4-ab79-c91f86606d74 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.440589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add818c-1af8-46bf-805b-790faf25ee2f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.454213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e59e5-150c-4043-981a-dd77413e7f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.455154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e59e5-150c-4043-981a-dd77413e7f19 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.455520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e59e5-150c-4043-981a-dd77413e7f19 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.456246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e59e5-150c-4043-981a-dd77413e7f19 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.456913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e59e5-150c-4043-981a-dd77413e7f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:48.457219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e59e5-150c-4043-981a-dd77413e7f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.458654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e59e5-150c-4043-981a-dd77413e7f19 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.460262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:48.477434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 311/621] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:48.478861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181eabe-373a-44ab-8728-584a503fd259 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.479213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181eabe-373a-44ab-8728-584a503fd259 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.479554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181eabe-373a-44ab-8728-584a503fd259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.484174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.484783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.485062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.485423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.486175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:48.486480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.488177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.499586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2181eabe-373a-44ab-8728-584a503fd259 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:48.501150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2181eabe-373a-44ab-8728-584a503fd259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:48.512036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 311/622] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:38:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:48.513477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.616020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 312/623] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:48.617298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181eabe-373a-44ab-8728-584a503fd259 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.712708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5877da-a90b-4ab7-bdce-78dbdf10dde5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.729372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:48.745012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181eabe-373a-44ab-8728-584a503fd259 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.755486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1937cff-76c3-458b-8ac1-a605dc6da365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.756335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1937cff-76c3-458b-8ac1-a605dc6da365 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.756459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1937cff-76c3-458b-8ac1-a605dc6da365 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.756695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1937cff-76c3-458b-8ac1-a605dc6da365 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.764264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1937cff-76c3-458b-8ac1-a605dc6da365 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.766393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1937cff-76c3-458b-8ac1-a605dc6da365 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.779587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c1937cff-76c3-458b-8ac1-a605dc6da365 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:48.785669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:48.786689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:48.787639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 312/624] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:48.792585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe40fbb-0048-4027-8320-1b1cd46afe3f None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.793786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe40fbb-0048-4027-8320-1b1cd46afe3f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.794759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe40fbb-0048-4027-8320-1b1cd46afe3f None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.796350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 313/625] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:48.797071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe40fbb-0048-4027-8320-1b1cd46afe3f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.800779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.802092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.803249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.803728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.931355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe40fbb-0048-4027-8320-1b1cd46afe3f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.932441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.934532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.974804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f84c5c-eb65-4e19-983f-311d95a9a483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.975701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f84c5c-eb65-4e19-983f-311d95a9a483 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.976899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f84c5c-eb65-4e19-983f-311d95a9a483 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.977302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f84c5c-eb65-4e19-983f-311d95a9a483 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.978104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f84c5c-eb65-4e19-983f-311d95a9a483 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:48.978474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f84c5c-eb65-4e19-983f-311d95a9a483 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.979987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f84c5c-eb65-4e19-983f-311d95a9a483 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:49.014275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 314/626] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Oct 4 08:38:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:49.016147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a18cad-203f-4c66-b2ab-03a21c23e4ac None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.016675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a18cad-203f-4c66-b2ab-03a21c23e4ac None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.017195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a18cad-203f-4c66-b2ab-03a21c23e4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.035344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-46a18cad-203f-4c66-b2ab-03a21c23e4ac None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:49.036639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-46a18cad-203f-4c66-b2ab-03a21c23e4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:49.054384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 313/627] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:49.055163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.075351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.084356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.147925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 315/628] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:49.147925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a18cad-203f-4c66-b2ab-03a21c23e4ac None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.182543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30043072-d23c-4d7c-8a97-5ec066494c0f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.221663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.222300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.222576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.222843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.233121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:49.237807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:49.238786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:49.239707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 314/629] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:49.240918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.268211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a18cad-203f-4c66-b2ab-03a21c23e4ac None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.307680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2290615-dae6-491c-b67e-6d1a7b0e823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.308430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2290615-dae6-491c-b67e-6d1a7b0e823f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.308709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2290615-dae6-491c-b67e-6d1a7b0e823f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.308979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2290615-dae6-491c-b67e-6d1a7b0e823f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.319325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2290615-dae6-491c-b67e-6d1a7b0e823f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:49.320868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2290615-dae6-491c-b67e-6d1a7b0e823f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:49.334740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-b2290615-dae6-491c-b67e-6d1a7b0e823f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:49.336219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:49.337950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:49.338842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:49.338842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:49.338842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:49.338842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 316/630] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:49.338842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050d33-cea7-4954-b6d8-61c5247f2540 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.339154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050d33-cea7-4954-b6d8-61c5247f2540 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.340493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050d33-cea7-4954-b6d8-61c5247f2540 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.340863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 317/631] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:49.341561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050d33-cea7-4954-b6d8-61c5247f2540 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.360057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7daeb-3e39-4a8b-98d8-8920f8fa4f80 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.361789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.396871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51ddd-1580-4bd5-acfa-0cd802d7e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.397627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51ddd-1580-4bd5-acfa-0cd802d7e4be None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.397898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51ddd-1580-4bd5-acfa-0cd802d7e4be None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.398166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51ddd-1580-4bd5-acfa-0cd802d7e4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.408358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51ddd-1580-4bd5-acfa-0cd802d7e4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:49.409880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51ddd-1580-4bd5-acfa-0cd802d7e4be None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-39e51ddd-1580-4bd5-acfa-0cd802d7e4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:49.425194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:49.426389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:49.427334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:49.427334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:49.427334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:49.427334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 315/632] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:49.427573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae6291-c191-436e-957b-367f4294061c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.427809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae6291-c191-436e-957b-367f4294061c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.428113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae6291-c191-436e-957b-367f4294061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.446635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6fae6291-c191-436e-957b-367f4294061c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:49.447748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6fae6291-c191-436e-957b-367f4294061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:49.460799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050d33-cea7-4954-b6d8-61c5247f2540 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.493875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.494474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.494752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.495020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.495678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:49.496435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:49.497775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:49.505812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 318/633] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:49.506586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.521985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 316/634] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:49.522831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae6291-c191-436e-957b-367f4294061c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.595281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.595529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.625348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6947-0c75-4fc4-99cb-a8fc99e3298f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.631786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae6291-c191-436e-957b-367f4294061c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.664975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.665975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.666581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.666862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.666965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.667052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.667133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.667418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.667748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:49.668071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:49.668165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:49.668457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:49.669460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:49.669795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:49.688657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 319/635] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:49.688657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.688920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 317/636] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:49.688920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.795902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713a53-d2b6-40c6-b8bd-045ccb51c64c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.800137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ac6b8-550c-4c4d-8be4-d12d3d5b6c5a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.819683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.828586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.829214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.829489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.829754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.833584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.834232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.834498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.834766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.837460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:49.840629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:49.842155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:49.845862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:49.850602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:49.851558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 320/637] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:49.852736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:49.856222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:49.859741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:49.861012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:49.861012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:49.861012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:49.861012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 318/638] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:49.861012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:49.960624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72f36-ce31-4ab2-b45f-ce77f09fe05a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.965939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fad57c-10ad-4c6e-a661-02e772985bca None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:49.993966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:49.994608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:49.994868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:49.995136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:49.999389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.000054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.000330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.000593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.002018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.004022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.007223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.008661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:50.015433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:50.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:50.017448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 321/639] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:50.017950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:50.019650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:50.020634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 319/640] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:50.021924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.059449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.080973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.120314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ede4c-8382-4c40-bb61-fee0f8a6eae5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.123742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc391bf2-6127-4606-8098-bf699660ec3e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.164070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.164801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.165102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.165371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.168463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.169071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.169349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.169621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.170291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:50.170594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.171988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.175726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:50.180029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 320/641] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:50.180815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:50.180933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:50.181843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 322/642] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:50.182730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.279818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.281910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e5ce6-2da7-44f2-b6cb-07db0d9cd59d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.282805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb56dbc-3d57-41f1-972b-3908611a4edb None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.298076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.315503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.316066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.316342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.316625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.317322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:50.317648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.318667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.319061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.319271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.319546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.319812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.320480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:50.320781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.322141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.330970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 323/643] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:50.331750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.334396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 321/644] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:50.336267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.427257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f300f8-decd-4736-a029-815eae4444a3 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.432398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905fd28-b8db-4220-91c8-5526e412846d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.465004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.465604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.465836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.465970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.466132 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.466466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.466749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.466843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:50.467017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.467105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.468591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.477147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.478726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.479795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 324/645] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:50.480630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:50.498613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:50.499752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:50.500897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:50.500897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:50.500897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:50.504042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 322/646] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:50.505026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.515071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.519453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.581158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8520a4-9e21-424d-afc5-89d36b1ebdec None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.602790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc414d-a8d0-418d-bef2-3ee33827015c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.614242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.614849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.615126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.615396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.616073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:50.616375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.617736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.625970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 325/647] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:50.626667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.641218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.641939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.642313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.642677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.643512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=bf149f246c1644aa85c4c162a2e07984) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:50.643820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.645278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.651009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 323/648] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/role_assignments?user.id=bf149f246c1644aa85c4c162a2e07984&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:50.651804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.734885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.740075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08fd9-ed7c-44f1-88d9-5c93863bdd6a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.754833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.780744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da2cb4-c0f7-4e2b-8543-81f8452ee300 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:50.783201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.783861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.784183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.784459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] PATH_INFO: `/v3/users/glance` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.794545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.796045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:50.808501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:50.809969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 326/649] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:50.810920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.826300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:50.826959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.827236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.827510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/c76ff74c7e67426bb2b94a4bb1a2b9ec/roles/4460b00f7dab40cd8eb26f0b606b4101` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.844721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=c76ff74c7e67426bb2b94a4bb1a2b9ec, role_id=4460b00f7dab40cd8eb26f0b606b4101)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:50.846192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:50.861541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 324/650] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:38:50 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/c76ff74c7e67426bb2b94a4bb1a2b9ec/roles/4460b00f7dab40cd8eb26f0b606b4101 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:38:50.863222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:50.950442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cea565-a3e6-4db3-ae8c-e4d5dd01067c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.000057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d745839-bd75-484c-9a99-379805c2f21d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.008321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.009753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.010116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.010516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.011247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:51.011625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.013648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.028901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 327/651] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:51.029920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.037899 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.054062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.054969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.055392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.055836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.065064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.067152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.068719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:51.085831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:51.087051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:51.088282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:51.088282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:51.088282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:51.088282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 325/652] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:51.088282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.174798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74ea90-133a-4080-8dcc-85f2f6a11aac None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.201199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaf6f2-4a3d-4c27-9a7f-05899e15d5af None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.223870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.224817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.225104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.225376 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.226362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:51.226667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.228830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.237933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.238696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.239149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.239548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.247726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.249464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.250413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 328/653] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:51.251356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:51.262782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:51.263976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:38:51.265137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:51.267513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 326/654] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:51.267915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.309633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.351850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.375580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd0286-08f9-4429-b13e-9a560a06e0b2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.383652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec48079-25ba-4bdb-b083-119d897dbc52 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.415659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.416795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.417224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.417800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.427072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.427736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.428318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.428553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.429345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:51.429745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.431197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.443366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 329/655] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:51.449316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.504847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.507400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.559944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.578250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c1c61-e3a1-43fe-9d15-2526a4db2a50 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.624011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.625286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.625968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.626388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.627501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:51.628160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.629583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.635210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.640097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 327/656] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:51.641776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.654463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 330/657] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:51.655715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.751368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f93eb1-1116-41a2-be89-000acda54d90 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.783778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19eb44e-4945-4db7-8ea5-7428eef8df2d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.787812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.789012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.789498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.789966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.812301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.814520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.818374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:51.835875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:51.836918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:38:51.837941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:51.844371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.844913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.845327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.845496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 328/658] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:51.845705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.846714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.858666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.860600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:51.877823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:51.878830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:51.883961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:51.883961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:51.883961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:51.883961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 331/659] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:51.883961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.936216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.983413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1928f1-3429-499b-b1ad-e4b8c8b22e05 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.001221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481a2d1-18a6-459d-b8a0-a12ca73fb547 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.019985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.020983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.021558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.022020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.042516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.043132 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.043418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.043691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.044506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:52.044809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.046672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.072797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 332/660] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:52.072797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.097119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:52.102820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.104602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.191769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd9076-f05b-4ee9-a56a-5fefceba294f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.194924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:52.233297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.233535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 329/661] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:52.234155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.234803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.235289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.235752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.236721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:52.237329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.238883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.251878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 333/662] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:52.252986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.344198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:52.350146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa43a52-fd42-4890-b043-c80f07819dc0 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.364790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa79ad2-a119-43b8-abfa-ad47c420423b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.388863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.389764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.390358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.390834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.409130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.409769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.410047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.410318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.424876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.426613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:52.445257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:52.446282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:52.447202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:52.447202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:52.447202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:52.447202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 334/663] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:52.447922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.466840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.468814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.470287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:52.573937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:52.586434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fd93-781c-4dbf-a95f-416d485d9d92 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.586795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 330/664] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:52.588023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.619762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.621145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.621667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.622128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.629543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.631308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:52.643328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:52.644446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:52.645444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:52.647732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 335/665] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:52.648819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.708007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a045-41ab-4243-978e-e7f49dd5c5bb None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.731188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:52.758034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.758665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.758946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.759220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.759873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:52.760202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.761329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298ba7f-ed49-4347-a86c-9629b35d68a9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.761595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.776716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 331/666] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:52.777643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.803158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.803823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.804200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.804468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.816234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.817848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:52.835400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:52.836508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:52.837415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:52.837415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:52.837415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:52.837415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 336/667] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:52.838171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.863554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:52.916955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25e03f-8376-4162-a0d8-9590ab70e6c6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.956548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516fad0-64a3-4c1e-9c97-4fc2e9927042 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:52.960247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:52.960993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:52.961364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:52.961731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:52.962545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:52.962941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:52.967666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:52.983964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 332/668] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:52.983964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:52.991771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.000879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa4304-01ed-4e23-ad2e-385fe78d08a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.001608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa4304-01ed-4e23-ad2e-385fe78d08a7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.001949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa4304-01ed-4e23-ad2e-385fe78d08a7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.002277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa4304-01ed-4e23-ad2e-385fe78d08a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.010060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa4304-01ed-4e23-ad2e-385fe78d08a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:53.011653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa4304-01ed-4e23-ad2e-385fe78d08a7 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-07aa4304-01ed-4e23-ad2e-385fe78d08a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:53.024747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:53.025914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:53.027006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:53.032023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 337/669] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:53.032023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7a833-1f4f-4642-9068-43d70ab2cb3e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.032023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7a833-1f4f-4642-9068-43d70ab2cb3e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.032023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7a833-1f4f-4642-9068-43d70ab2cb3e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.035641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 338/670] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:53.036632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7a833-1f4f-4642-9068-43d70ab2cb3e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.093103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635445-1135-46c9-b36f-6fd2c38c11fd None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.110763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.129437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.130101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.130403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.130674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.131303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:53.131600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:53.132988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:53.144987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 333/671] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:53.145972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.147126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7a833-1f4f-4642-9068-43d70ab2cb3e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.182291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d3cc4-ab43-4341-b7d3-75e57353dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.182817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d3cc4-ab43-4341-b7d3-75e57353dbde None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.183095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d3cc4-ab43-4341-b7d3-75e57353dbde None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.183365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d3cc4-ab43-4341-b7d3-75e57353dbde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.184119 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d3cc4-ab43-4341-b7d3-75e57353dbde None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=557974bcc89c412f8d5c2e2039843f65) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:53.184424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d3cc4-ab43-4341-b7d3-75e57353dbde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:53.185860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d3cc4-ab43-4341-b7d3-75e57353dbde None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:53.191453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 339/672] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/role_assignments?user.id=557974bcc89c412f8d5c2e2039843f65&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:53.192657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183da9-9a42-4720-a793-d78c0c008008 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.192933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183da9-9a42-4720-a793-d78c0c008008 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.193247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183da9-9a42-4720-a793-d78c0c008008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.217045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-25183da9-9a42-4720-a793-d78c0c008008 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:53.218333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-25183da9-9a42-4720-a793-d78c0c008008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:53.252412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a9264-82e2-4da9-9f58-d9352efef245 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.275526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.292879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e489b8-6382-4081-972c-7ba79a1e3550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.293486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e489b8-6382-4081-972c-7ba79a1e3550 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.293763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e489b8-6382-4081-972c-7ba79a1e3550 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.294034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e489b8-6382-4081-972c-7ba79a1e3550 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.294727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e489b8-6382-4081-972c-7ba79a1e3550 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:53.295029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e489b8-6382-4081-972c-7ba79a1e3550 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:53.296509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e489b8-6382-4081-972c-7ba79a1e3550 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:53.306795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 340/673] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:53.306795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe450c9d-e056-4841-9c5b-71e6c7b4005e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.306795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe450c9d-e056-4841-9c5b-71e6c7b4005e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.306795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe450c9d-e056-4841-9c5b-71e6c7b4005e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.307542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 341/674] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:53.308339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe450c9d-e056-4841-9c5b-71e6c7b4005e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.313716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 334/675] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:53.315109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f07ce-3f51-44a7-a3a3-091f0e51a5ed None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.315493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f07ce-3f51-44a7-a3a3-091f0e51a5ed None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.315867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f07ce-3f51-44a7-a3a3-091f0e51a5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.335057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-654f07ce-3f51-44a7-a3a3-091f0e51a5ed None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:53.336476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-654f07ce-3f51-44a7-a3a3-091f0e51a5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:53.370198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.422692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 335/676] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:53.423663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f07ce-3f51-44a7-a3a3-091f0e51a5ed None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.478573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe450c9d-e056-4841-9c5b-71e6c7b4005e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.532054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.532780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.533064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.533336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.548125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:53.549598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:53.563684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:53.564689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:53.565597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:53.566037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f07ce-3f51-44a7-a3a3-091f0e51a5ed None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.566114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 342/677] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:53.572802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.604252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.604916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.605196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.605470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.606204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=349ee49c297749468e09c4aa3dc9c218) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:53.606510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:53.607977 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:53.615392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 336/678] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/role_assignments?user.id=349ee49c297749468e09c4aa3dc9c218&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:53.619851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.640215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.677678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.709906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e071d-0c45-45d4-bbc8-0fab9b867f67 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.765907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.766891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.767182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.767456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.768490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:53.768805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:53.770533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:53.775147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc48c0-a1c2-4115-b253-24e0167a6d73 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.790042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 343/679] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:53.793174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.830413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.831344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.831686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.832034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.854906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:53.865721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:53.866756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:38:53.867614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:53.872200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 337/680] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:53.873491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:53.928520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939c215-52d5-41b9-83cd-f0731b5450a2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.930753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.981125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:53.981986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:53.982458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:53.982871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:53.986926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:53.993806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2321-cc50-43e7-adaa-a048fad401ae None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:53.996101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:38:54.000864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:54.001967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:54.002973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 344/681] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:54.003687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.044004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.044907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.045323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.045728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.046587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=d877daa4c2c14d0c8d281aeebab27ee1) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:54.047028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.049001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.063913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 338/682] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:53 2022] GET /identity/v3/role_assignments?user.id=d877daa4c2c14d0c8d281aeebab27ee1&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:54.063913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.143040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b6fee-2029-4e96-b49a-ac3e388f8d20 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.188069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.189060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.189489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.189900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.190711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:54.191165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.192805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.213431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 345/683] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:54.219490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.268869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:54.301323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f2965-da29-4c7a-9496-4409c85c77de None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.343955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.344572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.344850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.345117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.345779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:54.346077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.347431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.356436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 339/684] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:54.362391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.367295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7df73-1e1c-46f5-9e19-0c6bf5feb426 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.383231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:54.416522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.417836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.418261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.418818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.433781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.436211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:54.455987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:54.457213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:54.458389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:54.458389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:54.458389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:54.461141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 346/685] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:54.467518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.504097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d33ff-ab55-485d-97e7-e4aa5ad5e87e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.592791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.593363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.593644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.593915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.597012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:54.620605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.622365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.626925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c72c51-c4c8-4fb4-8537-892aff1ebca6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:54.654838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:54.655876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:54.656801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:54.656801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:54.656801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:54.664445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 340/686] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:54.667960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.668826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.669369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.669831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.670211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.671002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:54.671507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.673083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.688540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 347/687] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:54.698654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.820684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:54.853676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06c5b-02af-4cc6-a0d0-15eb96933c54 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.886896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69580a6-e2a1-4909-a754-81c4ef4bfbcb None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.891120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.891731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.892040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.892309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.893838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:54.894144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.895648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.909765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 341/688] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:54.914559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.958215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.958853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.959123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.959385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.965780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:54.975403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:54.976861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:54.999081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:55.000524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:55.001470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 348/689] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:55.006604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:55.099941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac7cd1-1fb5-4cde-bdfc-f5041a97d5b1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:55.141506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21f2cd-d198-4a8e-aa72-acc6fde44eb8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:55.162101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:55.176281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:55.176976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:55.177253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:55.177519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:55.178243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:55.178543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:55.180069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:55.180885 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:55.181780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:55.182253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:55.182777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:55.198138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 349/690] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:38:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:55.199546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:55.201251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:55.208880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:38:55.214033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:55.214972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:38:55.216027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:55.218558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 342/691] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:55.224287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:55.271525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:55.317647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca58938-53db-4cca-a6a3-61bf343ef1ba None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:55.379010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:55.379613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:55.380007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:55.380379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:55.433981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02448b-bbd0-42e6-8097-eb755b965ce7 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:55.493127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:55.493740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:55.494017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:55.494286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:55.495014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:55.495315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:55.497535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:55.501330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:55.502773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:55.532172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 343/692] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Oct 4 08:38:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:38:55.541435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:55.604590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:55.629317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:55.688717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 350/693] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:55.694333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:55.724468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e10735-53ec-4d37-bb1b-991851ad493d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:55.786952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3063a1e-fc73-4b7b-9328-44f247d50b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:55.787566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3063a1e-fc73-4b7b-9328-44f247d50b4a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:55.787844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3063a1e-fc73-4b7b-9328-44f247d50b4a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:55.788141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3063a1e-fc73-4b7b-9328-44f247d50b4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:55.865578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf2238-ac9b-4621-ad8e-47aba072a718 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:55.916140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3063a1e-fc73-4b7b-9328-44f247d50b4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:55.917625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3063a1e-fc73-4b7b-9328-44f247d50b4a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:55.946286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d86be-e29b-4274-a8c3-1e7bddf4bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:55.946468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d86be-e29b-4274-a8c3-1e7bddf4bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:55.946708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d86be-e29b-4274-a8c3-1e7bddf4bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:55.946925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d86be-e29b-4274-a8c3-1e7bddf4bdb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:55.972576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d86be-e29b-4274-a8c3-1e7bddf4bdb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:55.976163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d86be-e29b-4274-a8c3-1e7bddf4bdb6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-178d86be-e29b-4274-a8c3-1e7bddf4bdb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:56.005363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:56.007141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:56.008282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:56.008282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:56.008282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:56.008282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 351/694] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:38:56.008282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be3c79-1e0d-470f-92df-a852236ca8f1 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.008282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be3c79-1e0d-470f-92df-a852236ca8f1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.008624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be3c79-1e0d-470f-92df-a852236ca8f1 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.014253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 352/695] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:56.014669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be3c79-1e0d-470f-92df-a852236ca8f1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.016690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.033478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.112735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 344/696] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:56.114473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e139b-323c-4312-92d8-e2efba5f12f2 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.114826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e139b-323c-4312-92d8-e2efba5f12f2 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.115160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e139b-323c-4312-92d8-e2efba5f12f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.139792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be3c79-1e0d-470f-92df-a852236ca8f1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:56.143721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-150e139b-323c-4312-92d8-e2efba5f12f2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:56.145934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-150e139b-323c-4312-92d8-e2efba5f12f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:56.183292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:56.184032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.184501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.184829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.185699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:56.186079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:56.187465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.202916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 353/697] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:56.203679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.224687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 345/698] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:56.226012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e139b-323c-4312-92d8-e2efba5f12f2 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.302030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.329110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.345518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c627-61ab-474b-b125-d21fbde0cb5c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:56.354128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e139b-323c-4312-92d8-e2efba5f12f2 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:56.385390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73687349-ecef-4bd3-a0db-31265a6011b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:56.386072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73687349-ecef-4bd3-a0db-31265a6011b4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.386474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73687349-ecef-4bd3-a0db-31265a6011b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.386895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73687349-ecef-4bd3-a0db-31265a6011b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.400334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:56.401818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.402585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.403073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.403179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73687349-ecef-4bd3-a0db-31265a6011b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:56.404930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73687349-ecef-4bd3-a0db-31265a6011b4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.412156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:56.414433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-73687349-ecef-4bd3-a0db-31265a6011b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:56.422401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:56.423369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:56.424295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:56.424295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:56.424295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:56.435959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 354/699] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:56.435959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884ab0e-f9c0-4058-9824-884ed8a7cf32 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.435959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884ab0e-f9c0-4058-9824-884ed8a7cf32 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.435959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884ab0e-f9c0-4058-9824-884ed8a7cf32 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.435959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 355/700] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:56.436426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:56.437757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:56.438763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:38:56.439186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884ab0e-f9c0-4058-9824-884ed8a7cf32 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.439266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 346/701] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:56.444917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.570130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea9875-126a-44f3-b3bf-5a53a8286053 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:56.582597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884ab0e-f9c0-4058-9824-884ed8a7cf32 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:56.601714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.605104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5c184-89ca-450c-9aba-11bf98193e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:56.605727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5c184-89ca-450c-9aba-11bf98193e5c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.606003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5c184-89ca-450c-9aba-11bf98193e5c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.606320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5c184-89ca-450c-9aba-11bf98193e5c None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.607015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5c184-89ca-450c-9aba-11bf98193e5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:56.607367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5c184-89ca-450c-9aba-11bf98193e5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:56.608875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5c184-89ca-450c-9aba-11bf98193e5c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.617122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.621354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:56.622066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.622347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.622619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.623278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 347/702] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:56.624519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b280-3430-4073-9d6d-037c51ba0af8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.625218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b280-3430-4073-9d6d-037c51ba0af8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.625490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b280-3430-4073-9d6d-037c51ba0af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.628180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:56.628503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:56.630276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.642773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 356/703] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:56.643341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b833b280-3430-4073-9d6d-037c51ba0af8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:56.643571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.644584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b833b280-3430-4073-9d6d-037c51ba0af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:56.711752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 348/704] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:56.712455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833b280-3430-4073-9d6d-037c51ba0af8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.758546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8824973-ef95-478d-a57c-cd7f2693ac3a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:56.812196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17202e1-5a21-494a-a679-bb1c0279049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:56.812994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17202e1-5a21-494a-a679-bb1c0279049f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.813277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17202e1-5a21-494a-a679-bb1c0279049f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.813549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17202e1-5a21-494a-a679-bb1c0279049f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.825824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17202e1-5a21-494a-a679-bb1c0279049f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:56.827194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17202e1-5a21-494a-a679-bb1c0279049f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.838491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833b280-3430-4073-9d6d-037c51ba0af8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-e17202e1-5a21-494a-a679-bb1c0279049f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:38:56.841762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:56.842723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:38:56.843636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:38:56.843636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:38:56.843636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:56.843636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 357/705] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:56.845240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a56d0c-d4a9-4d8c-ace4-615e15b95d2c None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.845240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a56d0c-d4a9-4d8c-ace4-615e15b95d2c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.845240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a56d0c-d4a9-4d8c-ace4-615e15b95d2c None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.850151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 358/706] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:56.850151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a56d0c-d4a9-4d8c-ace4-615e15b95d2c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:56.878040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.885098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:56.888464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbff74c-26c3-484b-95bd-58915883a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:56.889126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff74c-26c3-484b-95bd-58915883a91b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.889408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff74c-26c3-484b-95bd-58915883a91b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.889679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff74c-26c3-484b-95bd-58915883a91b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.890599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbff74c-26c3-484b-95bd-58915883a91b None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=bf149f246c1644aa85c4c162a2e07984) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:56.890864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbff74c-26c3-484b-95bd-58915883a91b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:56.892524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbff74c-26c3-484b-95bd-58915883a91b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:56.908029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 349/707] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/role_assignments?user.id=bf149f246c1644aa85c4c162a2e07984&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:56.908029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232ea2-276a-4e09-a7ba-d5fb25dba8de None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:56.908029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232ea2-276a-4e09-a7ba-d5fb25dba8de None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:56.908029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232ea2-276a-4e09-a7ba-d5fb25dba8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:56.928051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d4232ea2-276a-4e09-a7ba-d5fb25dba8de None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:56.929290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d4232ea2-276a-4e09-a7ba-d5fb25dba8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:56.988622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a56d0c-d4a9-4d8c-ace4-615e15b95d2c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.022453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 350/708] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:57.023393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4232ea2-276a-4e09-a7ba-d5fb25dba8de None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.043048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.043857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.046003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.046283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.056298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.058444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:38:57.072377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:57.073401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:38:57.074307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:57.077070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 359/709] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:57.078288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.136029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4232ea2-276a-4e09-a7ba-d5fb25dba8de None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.162136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:57.173919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.174529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.174809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.175076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.175716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:57.176059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.177457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.193802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 351/710] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:57.193802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.194685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:57.208579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9dc33-5a43-4594-9488-6f9c08399ee8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.248951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.249943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.250559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.251031 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] PATH_INFO: `/v3/services/network` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.252378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.253984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:57.268795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:57.269880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:38:57.270840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:57.275929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 360/711] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:57.275929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.301795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d4cff-af84-4c00-9875-4a3e8dcd3f5b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.336929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.337457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.337730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.337996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.338675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:57.338975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.340620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.353032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 352/712] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:57.353841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.390504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3c41-b78d-4e7e-a6f6-f236a4aba150 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.397685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:57.428648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.429606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.430098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.430672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.431322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.432942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.463913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75dda2-0b8c-4c79-a336-69ce63089785 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.464461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 361/713] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:38:57 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:38:57.464461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.479853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:57.497144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.498124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.498561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.499021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.499933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] RBAC: Adding query filter params (name=network) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:57.500379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.502523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.508350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 353/714] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:57.509671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.594007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12cbbe-d18d-4ea5-9f94-870a049137c7 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.646370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.647024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.647298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.647570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.648503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.649729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.660109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b445324-b13e-4c6b-bfe1-2987835b860f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:38:57.661096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:38:57.662111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:38:57.662969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:38:57.668217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 362/715] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:38:57.670057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.675443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:57.732420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.732748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.733574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.733574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.734019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:57.734294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.736637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.743215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 354/716] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/role_assignments?user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:57.744018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.779724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:57.812760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de909d-ef24-4ab9-876d-cbf474804e3f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.852483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.853033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.853309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.853581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.854289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] RBAC: Adding query filter params (type=network) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:57.854594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.856047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.862886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 363/717] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:57.889280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524a2e6-2dc5-4a0b-b606-90ffc9ac70bd None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:57.931613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdd614-2a11-4ee4-a940-2fc0ea1cc1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:57.932532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdd614-2a11-4ee4-a940-2fc0ea1cc1d0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.932960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdd614-2a11-4ee4-a940-2fc0ea1cc1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.933376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdd614-2a11-4ee4-a940-2fc0ea1cc1d0 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.934183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdd614-2a11-4ee4-a940-2fc0ea1cc1d0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:57.934665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdd614-2a11-4ee4-a940-2fc0ea1cc1d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:57.936283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdd614-2a11-4ee4-a940-2fc0ea1cc1d0 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:57.942296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 355/718] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:57.953284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:58.081133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b5245-6ea6-4e74-8fc0-7c809449ea4e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:58.115338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27c069-49fb-413e-84e4-35bce7188e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:58.115967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27c069-49fb-413e-84e4-35bce7188e55 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:58.116184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27c069-49fb-413e-84e4-35bce7188e55 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:58.116642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27c069-49fb-413e-84e4-35bce7188e55 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:58.117248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27c069-49fb-413e-84e4-35bce7188e55 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:38:58.117550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27c069-49fb-413e-84e4-35bce7188e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:58.118923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27c069-49fb-413e-84e4-35bce7188e55 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:58.124439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 364/719] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:38:59.424601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a449-cd6c-4a1d-b838-86efe7b45644 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:59.425623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a449-cd6c-4a1d-b838-86efe7b45644 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:59.426103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9a449-cd6c-4a1d-b838-86efe7b45644 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:59.427918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 356/720] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:59.433418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2520-f002-4334-b9e6-09169ac16c60 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:59.434195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2520-f002-4334-b9e6-09169ac16c60 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:59.434690 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2520-f002-4334-b9e6-09169ac16c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:59.446678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:59.466526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d85d2520-f002-4334-b9e6-09169ac16c60 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:59.468044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d85d2520-f002-4334-b9e6-09169ac16c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:59.669501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 365/721] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:39:00.031980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a449-cd6c-4a1d-b838-86efe7b45644 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:00.076532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:00.170606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9a449-cd6c-4a1d-b838-86efe7b45644 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:00.216046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:00.216693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.216969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.217240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.231666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:00.236469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:00.237451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:00.238334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:00.242505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 357/722] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:00.243953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2520-f002-4334-b9e6-09169ac16c60 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:00.417945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2520-f002-4334-b9e6-09169ac16c60 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:00.456609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:00.461792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97432692-6387-46e6-989d-9426df77506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:00.462485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432692-6387-46e6-989d-9426df77506e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.468523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432692-6387-46e6-989d-9426df77506e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.468833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97432692-6387-46e6-989d-9426df77506e None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.469537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432692-6387-46e6-989d-9426df77506e None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:00.469887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432692-6387-46e6-989d-9426df77506e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:00.471451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97432692-6387-46e6-989d-9426df77506e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:00.481042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 366/723] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:00.486536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:00.602811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d2f45-64b2-4f0c-aa5b-49d2a1dfb769 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.603171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d2f45-64b2-4f0c-aa5b-49d2a1dfb769 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.603347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d2f45-64b2-4f0c-aa5b-49d2a1dfb769 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.604975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 367/724] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:00.612418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfad8-5127-4e4e-852b-47dbd60f3f36 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.612695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfad8-5127-4e4e-852b-47dbd60f3f36 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.612964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfad8-5127-4e4e-852b-47dbd60f3f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.643601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-305dfad8-5127-4e4e-852b-47dbd60f3f36 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:00.646712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-305dfad8-5127-4e4e-852b-47dbd60f3f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:00.656299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:00.665300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c0b2-a677-4315-bf3e-e1d7fbff0e18 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:00.710525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb16b4-ec4c-49d9-99e2-5c289759bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:00.711157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb16b4-ec4c-49d9-99e2-5c289759bc49 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.711431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb16b4-ec4c-49d9-99e2-5c289759bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.711749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb16b4-ec4c-49d9-99e2-5c289759bc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.722742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb16b4-ec4c-49d9-99e2-5c289759bc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:00.724345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb16b4-ec4c-49d9-99e2-5c289759bc49 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-fbdb16b4-ec4c-49d9-99e2-5c289759bc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:00.744383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:00.745555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:00.746999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:00.746999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:00.746999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:00.748726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 358/725] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:00.750046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e93bc6-94ee-46fc-8d9c-2068ac39a10a None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.750457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e93bc6-94ee-46fc-8d9c-2068ac39a10a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.750878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e93bc6-94ee-46fc-8d9c-2068ac39a10a None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.752305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 368/726] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:39:00.752621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 359/727] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:00.757068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfad8-5127-4e4e-852b-47dbd60f3f36 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:00.758341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d32db-c8ff-45fb-baab-ac6984c4c760 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.758849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d32db-c8ff-45fb-baab-ac6984c4c760 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.759330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d32db-c8ff-45fb-baab-ac6984c4c760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.781229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8e7d32db-c8ff-45fb-baab-ac6984c4c760 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:00.782831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8e7d32db-c8ff-45fb-baab-ac6984c4c760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:00.859982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 360/728] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:39:00.861311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9932a-111c-46af-aefa-414aedcb0ede None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.861600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9932a-111c-46af-aefa-414aedcb0ede None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.862001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9932a-111c-46af-aefa-414aedcb0ede None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.863395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 361/729] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:00.868585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534965b-97f8-4011-8425-5252ccbbf1b5 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.868866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534965b-97f8-4011-8425-5252ccbbf1b5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.869134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534965b-97f8-4011-8425-5252ccbbf1b5 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.877091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 362/730] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:00.877091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c332f0-52f6-4ba7-bb76-a423fe36dba3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.877091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c332f0-52f6-4ba7-bb76-a423fe36dba3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.877091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c332f0-52f6-4ba7-bb76-a423fe36dba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.886477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfad8-5127-4e4e-852b-47dbd60f3f36 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:00.895824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-82c332f0-52f6-4ba7-bb76-a423fe36dba3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:00.897969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-82c332f0-52f6-4ba7-bb76-a423fe36dba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:00.915237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:00.928598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065b6cd-3c1e-4988-8d03-40ef9b702673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:00.929527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b6cd-3c1e-4988-8d03-40ef9b702673 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.929809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b6cd-3c1e-4988-8d03-40ef9b702673 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.930068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b6cd-3c1e-4988-8d03-40ef9b702673 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.930711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b6cd-3c1e-4988-8d03-40ef9b702673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:00.931009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b6cd-3c1e-4988-8d03-40ef9b702673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:00.932388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b6cd-3c1e-4988-8d03-40ef9b702673 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:00.948967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 369/731] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:00.950350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beae1b8-8341-4e32-a619-b62ca9d2a689 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.950642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beae1b8-8341-4e32-a619-b62ca9d2a689 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.950916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beae1b8-8341-4e32-a619-b62ca9d2a689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.973378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2beae1b8-8341-4e32-a619-b62ca9d2a689 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:00.974590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2beae1b8-8341-4e32-a619-b62ca9d2a689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:01.004005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 363/732] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:39:01.004005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c332f0-52f6-4ba7-bb76-a423fe36dba3 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.013181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.067246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 370/733] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:39:01.068328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beae1b8-8341-4e32-a619-b62ca9d2a689 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.217756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beae1b8-8341-4e32-a619-b62ca9d2a689 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.245266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.277969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.278937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.279394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.279838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.293846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:01.294453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c332f0-52f6-4ba7-bb76-a423fe36dba3 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:01.299347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:01.300493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:39:01.301649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:01.303820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 371/734] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:39:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:01.304791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.340364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.341273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.341794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.342067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] PATH_INFO: `/v3/users/nova` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.349282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:01.350737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:01.362135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:01.363152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 364/735] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:39:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:01.364166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.416503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1c2f5-8253-42d3-8151-0104e59ccdc3 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.432679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.452240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.452843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.453169 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.453438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.454094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:01.455451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:01.473516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11648f3-842b-463d-b4a4-8d44581ff94d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.474579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 372/736] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:01 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:01.475962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.517866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.522069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.523091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.523551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.524033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.525040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:01.525488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:01.527643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:01.541208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 365/737] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:39:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:01.542056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.625801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbc4b1-293d-4cd6-b0aa-381fda3a872d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.669154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.669522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.669801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.670071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.670745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:01.672106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:01.677459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95306aa-e4a7-49d4-b724-1e8d0cdb9a71 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.689516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 373/738] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:01 2022] POST /identity/v3/services => generated 240 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:01.690326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.703225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.727297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.728767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.729947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.730723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.732018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:01.732677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:01.735183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:01.762040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 366/739] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:39:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:01.767922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.812370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.820193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5546b-5319-4553-8623-694d3e4b7e3c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.857287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.858348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.858348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.858520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.870587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:01.876447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:01.877581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:01.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:01.882570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 374/740] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:01.883423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:01.908295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acc229-7561-4866-8f12-5851117b5be2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.952484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.953516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.953857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.954384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.968652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:01.970116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:01.989901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:01.990918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:01.991810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:01.991810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:01.991810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:01.996183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 367/741] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:01.997060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.006385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e155575-8bdc-4801-9362-a98b32f7c336 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.036164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.047700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.048696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.049219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.049664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.062676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.064510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:02.082636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:02.084170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:02.085135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:02.085135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:02.085135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:02.085934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 375/742] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:02.086516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.101372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.131642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fc77d-1916-440e-8f81-ea7b9e0e0e22 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.166303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.167300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.167799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.168290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.169173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:02.169676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.171230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.185069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 368/743] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:02.186267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.208379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f095e0-9107-4614-91f8-ee4778e0e0d4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.248469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.249105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.249386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.249656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.250305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:02.250623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.251993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.268818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 376/744] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:02.269743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.288785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.300021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ca487-4c27-441a-99fa-d8b96630405f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.339189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.343609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.345056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.346472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.346922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.347773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:02.348235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.349892 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.366135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 369/745] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:02.367672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.379876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ebf19-c82b-417b-b6ae-cb80065f807d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.421428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.422232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.422614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.423341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.435384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.437096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:02.454048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:02.455289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:02.456471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:02.456471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:02.456471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:02.462307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 377/746] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:02.464098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.503755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43245649-0c4a-4ec1-9d13-ca04e715d026 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.549503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.551105 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.551828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.552283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.563148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.564834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:02.580336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:02.581551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:02.582763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:02.590390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 370/747] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:02.590390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.601047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98869-4976-4335-83fe-3b89bd742842 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.610568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.620765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.640359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.641563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.641563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.641931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.651160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.653709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:02.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:02.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:02.675141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:02.677740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 378/748] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:02.679464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.724784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cb109-8234-4db7-8439-ecdc5c926bb6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.762262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.763120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.763491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.763965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.764715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:02.765106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.766699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.784329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 371/749] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:02.785421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.789761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3ca16-2398-4ccb-9bef-b8ae440ce258 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.824109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.824721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.824997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.825269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.826003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:02.826307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.827720 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.845617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 379/750] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:02.846380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.860663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.885085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.897185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f124b-99b0-4a3c-8823-d3cd9f070054 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.933688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.934756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.935317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.935859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.936797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:02.937297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:02.939200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:02.953716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a41d1d-e0ee-4ebe-9714-4792b984f2da None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.961330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 372/751] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:02.962656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.997881 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.998505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.998778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.999042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.006907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.009713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:03.030997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:03.032053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:39:03.033030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:03.040178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 380/752] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:03.041408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.077477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5838-2e8c-4bfb-8d89-c76042315b6e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:03.111014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.118565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:03.119572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.120104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.120569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.143072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.149515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082e859-8dff-4beb-b183-ee95d4188044 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:03.185089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:03.185826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.186301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.186408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/bf149f246c1644aa85c4c162a2e07984/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.188390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=bf149f246c1644aa85c4c162a2e07984, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.189985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.200634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.202392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.212138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 381/753] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:39:03 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/bf149f246c1644aa85c4c162a2e07984/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:39:03.213510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.329268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 373/754] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:03.335419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.346090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa21072-9e10-4a14-a580-e11cc935b764 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:03.370366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.421434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:03.422063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.422335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.422598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.423336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:03.423635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.425185 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.449049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d8b6a-d45e-48f7-abd3-5f70c0d9c221 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:03.464994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 382/755] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Oct 4 08:39:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:03.470180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.486589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:03.487489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.487939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.488442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.493185 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.504623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.506446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:03.525229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:03.526540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:03.527658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:03.527658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:03.527658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:03.534553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 374/756] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:03.535733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.599176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c28c1f-e0c8-43cd-84d4-1eb17db967bf None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:03.629676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.644437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74081b1-ef14-4b0e-8ef6-b6287e4d2576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:03.645247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74081b1-ef14-4b0e-8ef6-b6287e4d2576 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.645625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74081b1-ef14-4b0e-8ef6-b6287e4d2576 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.645992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74081b1-ef14-4b0e-8ef6-b6287e4d2576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.662505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fd7f1-de63-45bd-8c9e-a0dfb46e694e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:03.725988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:03.726608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.726886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.727159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.727804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:03.728129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.729542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.756750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 375/757] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:03.763372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.770502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74081b1-ef14-4b0e-8ef6-b6287e4d2576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.771905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74081b1-ef14-4b0e-8ef6-b6287e4d2576 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.800544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.917996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf1f5-3306-41f2-ad12-81eed0976c29 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:03.920632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 383/758] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 08:39:03.922063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3943e39-2e65-4239-93a3-56a0ec71c913 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.922506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3943e39-2e65-4239-93a3-56a0ec71c913 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.922922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3943e39-2e65-4239-93a3-56a0ec71c913 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.924616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 384/759] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:03.933353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3943e39-2e65-4239-93a3-56a0ec71c913 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.965381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db163ea-5e0a-4137-b318-991668135f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:03.966585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db163ea-5e0a-4137-b318-991668135f6a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.967015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db163ea-5e0a-4137-b318-991668135f6a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.967432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db163ea-5e0a-4137-b318-991668135f6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.978095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db163ea-5e0a-4137-b318-991668135f6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.980723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db163ea-5e0a-4137-b318-991668135f6a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-3db163ea-5e0a-4137-b318-991668135f6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:03.995660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:03.997134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:03.998230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:04.000520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 376/760] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:04.001811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67395d87-a7df-4f81-b131-0df7bafffbec None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.002219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67395d87-a7df-4f81-b131-0df7bafffbec None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.002686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67395d87-a7df-4f81-b131-0df7bafffbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.011547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.027429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-67395d87-a7df-4f81-b131-0df7bafffbec None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:04.029488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-67395d87-a7df-4f81-b131-0df7bafffbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:04.071577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3943e39-2e65-4239-93a3-56a0ec71c913 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.089231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.108207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 377/761] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:04.108207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67395d87-a7df-4f81-b131-0df7bafffbec None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.113268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f60126-92c9-4548-b6d0-5d69f63fe405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.114249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f60126-92c9-4548-b6d0-5d69f63fe405 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.114703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f60126-92c9-4548-b6d0-5d69f63fe405 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.115169 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f60126-92c9-4548-b6d0-5d69f63fe405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.126652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f60126-92c9-4548-b6d0-5d69f63fe405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.128510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f60126-92c9-4548-b6d0-5d69f63fe405 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-19f60126-92c9-4548-b6d0-5d69f63fe405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:04.144328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:04.145386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:04.146300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:04.146300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:04.146300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:04.148384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 385/762] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:04.149972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407346d-85fb-4b82-86dd-256959473b1a None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.150413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407346d-85fb-4b82-86dd-256959473b1a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.150838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407346d-85fb-4b82-86dd-256959473b1a None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.152500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 386/763] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:04.153439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407346d-85fb-4b82-86dd-256959473b1a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.225667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67395d87-a7df-4f81-b131-0df7bafffbec None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.265817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407346d-85fb-4b82-86dd-256959473b1a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.270861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3849c-b983-42bf-96e4-73abfb8e93f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.271485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3849c-b983-42bf-96e4-73abfb8e93f5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.271762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3849c-b983-42bf-96e4-73abfb8e93f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.272078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3849c-b983-42bf-96e4-73abfb8e93f5 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.272775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3849c-b983-42bf-96e4-73abfb8e93f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:04.273083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3849c-b983-42bf-96e4-73abfb8e93f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.274525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3849c-b983-42bf-96e4-73abfb8e93f5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.276746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.292884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 378/764] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:04.294927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72712b35-442c-49aa-97dc-8c93221f6e17 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.295215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72712b35-442c-49aa-97dc-8c93221f6e17 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.295542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72712b35-442c-49aa-97dc-8c93221f6e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.305143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.305901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.306246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.306575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.307285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:04.307642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.309098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.319313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-72712b35-442c-49aa-97dc-8c93221f6e17 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:04.320794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-72712b35-442c-49aa-97dc-8c93221f6e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:04.321353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 387/765] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:04.322232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.354089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.413970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 379/766] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:04.414914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72712b35-442c-49aa-97dc-8c93221f6e17 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.442359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357ea3f-088f-4d53-b13f-c221311f025b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.478342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.479198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.479705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.480176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.481168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=3e23eac1e2b04f59bbdac1646da01cbe) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:04.481674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.483283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.489622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 388/767] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/role_assignments?user.id=3e23eac1e2b04f59bbdac1646da01cbe&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:04.490712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.516423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72712b35-442c-49aa-97dc-8c93221f6e17 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.556579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.580761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.581527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.581819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.582091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.597177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.598715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.601755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.610937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b53d92-41d6-4286-89c4-d846cacf220e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:04.626358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:04.627464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:04.628416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:04.636563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 380/768] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:04.641985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.657875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.659262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.659744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.660253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.661197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.662947 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:04.673027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:04.674145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:39:04.675004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:04.679380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 389/769] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:04.689761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.821505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272c566-cc4a-452d-abb6-2694df7da21b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.846253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9c70b-7393-4e64-ab1e-f7864107174e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.857950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.858567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.858844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.859132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.859934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:04.860243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.861847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.879535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 381/770] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:04.880528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.891156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.892521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.893025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.893885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.895047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:04.895650 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.897360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.907924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 390/771] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:04.907924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.928472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.949780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:05.025777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7640c2-6643-4a30-bf0d-f8c648d08949 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.029432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291caab3-d487-4974-a9f5-9624eaced009 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.067552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.068631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.069055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.069466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.070527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:05.070643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.071086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.071430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.071859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.072200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] PATH_INFO: `/v3/services/network` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.072887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.073353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.074428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.080255 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 391/772] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/role_assignments?user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:05.081488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:05.089208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:05.090491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:39:05.091354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:05.096003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 382/773] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:05.099813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:05.237777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:05.266585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061fa1f-3a06-4960-bd47-efc3b32e3115 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.287713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f8f11-ac63-4db6-87e9-45eee42406ea None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.306034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:05.307157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.308744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.309438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.309954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.311370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:05.311880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.313568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.322378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 392/774] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Oct 4 08:39:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:05.328603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:05.356372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.356986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.357260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.357545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.358172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] RBAC: Adding query filter params (name=network) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:05.358470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.359836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.368809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 383/775] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:05.375759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:05.482783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd9d8-9fb2-482c-88cf-615031b24af6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.548594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257176ad-88b8-42ad-9eba-014016768b56 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.558258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.559044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.559315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.559596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.564412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] RBAC: Adding query filter params (interface=public, service_id=65f1493bcbae4e2c8f930deff5b4daee, region_id=RegionOne) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:05.564412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.564412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.580491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 393/776] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:39:05 2022] GET /identity/v3/endpoints?service_id=65f1493bcbae4e2c8f930deff5b4daee&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:05.587098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:05.627257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:05.652827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.653508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.653704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.653968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.654598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] RBAC: Adding query filter params (type=network) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:05.654891 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.659153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.672882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 384/777] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:05.679107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:05.739153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:05.750291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad950c92-3bea-4da1-8e52-1a874271ceb4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.793368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627800c1-7908-4a78-819e-9243f6c257c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.794102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627800c1-7908-4a78-819e-9243f6c257c8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.794381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627800c1-7908-4a78-819e-9243f6c257c8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.794653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627800c1-7908-4a78-819e-9243f6c257c8 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.795268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627800c1-7908-4a78-819e-9243f6c257c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.797811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627800c1-7908-4a78-819e-9243f6c257c8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.804255 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 394/778] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:05 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:05.806345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f176e-ccae-45fa-9b51-968d092ec361 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.806675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f176e-ccae-45fa-9b51-968d092ec361 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.807240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f176e-ccae-45fa-9b51-968d092ec361 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.813530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 395/779] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:05.815516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d78e-5c7f-43fa-8ec8-70e5b5e3bc4b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.815804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d78e-5c7f-43fa-8ec8-70e5b5e3bc4b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.816102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d78e-5c7f-43fa-8ec8-70e5b5e3bc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.831420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52d1ef-7cf7-4441-ae57-847ea8ca018b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:05.838961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c5a6d78e-5c7f-43fa-8ec8-70e5b5e3bc4b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:05.840899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c5a6d78e-5c7f-43fa-8ec8-70e5b5e3bc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:05.869770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:05.870398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:05.870674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:05.871247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:05.872004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] RBAC: Adding query filter params (interface=public, service_id=16487b139dd5424796091b652452a6dd, region_id=RegionOne) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:05.872314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:05.873701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:05.882580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 385/780] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:39:05 2022] GET /identity/v3/endpoints?service_id=16487b139dd5424796091b652452a6dd&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:05.889009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:05.931620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 396/781] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:06.053193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:06.056250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61247f07-e57f-497e-bd0b-debb6b238848 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:06.109704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:06.110291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:06.110569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:06.110839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:06.111457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:06.112894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:06.124121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 386/782] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:05 2022] GET /identity/v3/services => generated 772 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:06.233378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d78e-5c7f-43fa-8ec8-70e5b5e3bc4b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:06.236781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:06.347418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d78e-5c7f-43fa-8ec8-70e5b5e3bc4b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:06.384820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:06.385816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:06.386295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:06.386760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:06.397954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:06.403150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:06.404165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:06.405038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:06.407728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 397/783] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:06.413921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:06.556201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46da0-e45c-4a27-858b-52fa80b55497 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:06.601032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:06.602150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:06.602634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:06.603103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:06.604017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:06.604524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:06.606119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:06.616604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 387/784] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:06.622985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:06.690588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:06.732730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1f2ac-ee66-4ca6-8809-f44114b3fe3a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:06.770233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:06.771732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:06.772261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:06.772733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:06.784366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:06.786282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:06.804311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:06.805975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:06.808617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:06.808617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:06.808617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:06.810707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 398/785] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:06.816691 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:06.846142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:06.959853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd5a14-a52b-49b3-9b03-7f3e06c554f8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:07.012038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:07.012670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:07.012928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:07.013194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:07.013836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:07.014132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:07.015508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:07.027513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 388/786] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:07.032987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:07.183530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2c929-8c50-4ccf-8421-6c3c78a2b58b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:07.200579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:07.219295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d8827-d0e3-4ebc-b47c-5e5a1acacea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:07.219851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8827-d0e3-4ebc-b47c-5e5a1acacea5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:07.220154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8827-d0e3-4ebc-b47c-5e5a1acacea5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:07.220424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8827-d0e3-4ebc-b47c-5e5a1acacea5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:07.227862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d8827-d0e3-4ebc-b47c-5e5a1acacea5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:07.229331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d8827-d0e3-4ebc-b47c-5e5a1acacea5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-504d8827-d0e3-4ebc-b47c-5e5a1acacea5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:07.250049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:07.251247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:07.252437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 399/787] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:39:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:07.259016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:07.400141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef03127-a5b9-41b6-ab53-30f44317302d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:07.400426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef03127-a5b9-41b6-ab53-30f44317302d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:07.400699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef03127-a5b9-41b6-ab53-30f44317302d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:07.402749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 400/788] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:07.407651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b3787-272e-48d5-b6ee-56aa3cdde7c6 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:07.407954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b3787-272e-48d5-b6ee-56aa3cdde7c6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:07.408227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b3787-272e-48d5-b6ee-56aa3cdde7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:07.412452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:07.440097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c48b3787-272e-48d5-b6ee-56aa3cdde7c6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:07.441294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c48b3787-272e-48d5-b6ee-56aa3cdde7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:07.444911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111d062-08c6-49a3-ab5f-1673258ee679 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:07.506967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:07.507500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:07.507779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:07.508076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:07.508808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:07.509119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:07.510551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:07.545718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 389/789] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:39:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:07.551925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:07.578064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 401/790] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:07.717860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc74f5-ab84-46c8-acca-2d720aa3a2a3 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:07.766692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:07.767218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:07.767494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:07.767760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:07.792237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:07.861448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:07.862907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:08.000098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 390/791] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:08.005351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b3787-272e-48d5-b6ee-56aa3cdde7c6 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.032322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.097375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:08.152051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b3787-272e-48d5-b6ee-56aa3cdde7c6 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.159548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e46e0-2cbf-4131-8961-79b73807e1de None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.162469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:08.192623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75765c6-06db-41c1-ba5a-3002f3d0219e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.193640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75765c6-06db-41c1-ba5a-3002f3d0219e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.194142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75765c6-06db-41c1-ba5a-3002f3d0219e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.194693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75765c6-06db-41c1-ba5a-3002f3d0219e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.200328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67710aa-05ed-4a36-b6c5-fcc5b0b19425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.200958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67710aa-05ed-4a36-b6c5-fcc5b0b19425 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.201232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67710aa-05ed-4a36-b6c5-fcc5b0b19425 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.201500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67710aa-05ed-4a36-b6c5-fcc5b0b19425 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.207481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75765c6-06db-41c1-ba5a-3002f3d0219e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:08.209512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75765c6-06db-41c1-ba5a-3002f3d0219e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:08.213368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67710aa-05ed-4a36-b6c5-fcc5b0b19425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-e67710aa-05ed-4a36-b6c5-fcc5b0b19425 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:08.219081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:08.220278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:39:08.221135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:08.223954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 391/792] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:08.225483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcb309-3ea3-4391-8c7c-b209b0f8fbbf None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.226118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcb309-3ea3-4391-8c7c-b209b0f8fbbf None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.226392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcb309-3ea3-4391-8c7c-b209b0f8fbbf None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.228525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 392/793] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:08.229229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcb309-3ea3-4391-8c7c-b209b0f8fbbf None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-a75765c6-06db-41c1-ba5a-3002f3d0219e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:08.233645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:08.234611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:08.235515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:08.235515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:08.235515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:08.243123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 402/794] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:08.245144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468d0c-debc-4972-826f-4a28e5d8110e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.246234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468d0c-debc-4972-826f-4a28e5d8110e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.246673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468d0c-debc-4972-826f-4a28e5d8110e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.269601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-69468d0c-debc-4972-826f-4a28e5d8110e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:08.271053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-69468d0c-debc-4972-826f-4a28e5d8110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:08.346030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcb309-3ea3-4391-8c7c-b209b0f8fbbf None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.349734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:08.359469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 403/795] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:08.360975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69468d0c-debc-4972-826f-4a28e5d8110e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.388122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.389398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.389818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.390229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.391199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:08.391759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:08.393533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:08.403485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 393/796] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:08.409201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.524474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570a657-4116-40cb-91c6-99339f8cc49b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.552433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:08.566903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.567798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.568255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.568648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.581549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:08.582968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69468d0c-debc-4972-826f-4a28e5d8110e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.583560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:08.602538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:08.603546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:08.604495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:08.604495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:08.604495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:08.606799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 394/797] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:08.612784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.639214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:08.655552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.656261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.656579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.656862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.657529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:08.657891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:08.659303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:08.690076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 404/798] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:08.695633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.740243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f9f26-8856-4262-867b-0737145885ba None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.782677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53220102-1e8c-4836-afb4-29d4e6346223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.783285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53220102-1e8c-4836-afb4-29d4e6346223 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.783559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53220102-1e8c-4836-afb4-29d4e6346223 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.783824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53220102-1e8c-4836-afb4-29d4e6346223 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.784506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53220102-1e8c-4836-afb4-29d4e6346223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:08.784801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53220102-1e8c-4836-afb4-29d4e6346223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:08.786288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53220102-1e8c-4836-afb4-29d4e6346223 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:08.798030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 395/799] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:08.799183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc03b5a-4c3b-49de-95b4-9101ea18a31d None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.799461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc03b5a-4c3b-49de-95b4-9101ea18a31d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.799731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc03b5a-4c3b-49de-95b4-9101ea18a31d None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.804096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 396/800] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:08.804096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ad05-91d6-4bd3-9a93-7c838698f338 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.804096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ad05-91d6-4bd3-9a93-7c838698f338 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.804096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8ad05-91d6-4bd3-9a93-7c838698f338 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.804096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 397/801] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:08.804673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ad05-91d6-4bd3-9a93-7c838698f338 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.820844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445652b9-1b99-47e0-bb0e-d47d547eced7 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.854674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.855223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.855498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.855766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.862277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:08.863698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:08.875117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:08.876294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:08.877184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 405/802] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:08.877679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:08.882981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:08.891699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:08.906671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8ad05-91d6-4bd3-9a93-7c838698f338 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:08.939361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390142d-0c98-4d49-8866-ef239b2c179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:08.940009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390142d-0c98-4d49-8866-ef239b2c179c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.940285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390142d-0c98-4d49-8866-ef239b2c179c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.940551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390142d-0c98-4d49-8866-ef239b2c179c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.950879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390142d-0c98-4d49-8866-ef239b2c179c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-2390142d-0c98-4d49-8866-ef239b2c179c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:08.955585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:08.956576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:08.957442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 398/803] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:08.958138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c806-cb79-4550-88f4-7590a07c6980 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:08.958138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c806-cb79-4550-88f4-7590a07c6980 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:08.958960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1c806-cb79-4550-88f4-7590a07c6980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:08.980001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b2d1c806-cb79-4550-88f4-7590a07c6980 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:08.981196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b2d1c806-cb79-4550-88f4-7590a07c6980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:08.983015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e62d1-c481-4b5b-969c-e85f021fbece None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.016657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba9462-47a4-42da-9435-d8501b0e6d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.017263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba9462-47a4-42da-9435-d8501b0e6d2b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.017533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba9462-47a4-42da-9435-d8501b0e6d2b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.017801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba9462-47a4-42da-9435-d8501b0e6d2b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.025409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba9462-47a4-42da-9435-d8501b0e6d2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.026900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba9462-47a4-42da-9435-d8501b0e6d2b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-bfba9462-47a4-42da-9435-d8501b0e6d2b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:09.037982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:09.038971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 406/804] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:39:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:09.040313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f0c0-c510-496b-bc9e-bf4c901e732f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.041042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f0c0-c510-496b-bc9e-bf4c901e732f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.041042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1f0c0-c510-496b-bc9e-bf4c901e732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.051380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 399/805] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:09.052925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1c806-cb79-4550-88f4-7590a07c6980 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.057855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c8f1f0c0-c510-496b-bc9e-bf4c901e732f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:09.059055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c8f1f0c0-c510-496b-bc9e-bf4c901e732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:09.122151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.129015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 407/806] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:09.129792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f0c0-c510-496b-bc9e-bf4c901e732f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.134233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.169040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1c806-cb79-4550-88f4-7590a07c6980 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.205460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118195e2-96be-4230-82e4-226007070398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.206156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118195e2-96be-4230-82e4-226007070398 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.206500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118195e2-96be-4230-82e4-226007070398 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.206844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118195e2-96be-4230-82e4-226007070398 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.207681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118195e2-96be-4230-82e4-226007070398 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:09.208071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118195e2-96be-4230-82e4-226007070398 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.209561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118195e2-96be-4230-82e4-226007070398 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.228049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 400/807] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:09.228049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118195e2-96be-4230-82e4-226007070398 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.261732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1f0c0-c510-496b-bc9e-bf4c901e732f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.306412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.307627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.308127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.309152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.310299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:09.310798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.313491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.327319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 408/808] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:09.327319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.348788 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118195e2-96be-4230-82e4-226007070398 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.375865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.385515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.386400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.386843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.387316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.388214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:09.388659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.389890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.391441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.408467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 401/809] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:09.409469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.431028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b503e-063e-4d7c-b106-5d85f5501256 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.464887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.465491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.465771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.466042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.466749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=bf149f246c1644aa85c4c162a2e07984) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:09.467053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.468506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.474363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 409/810] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/role_assignments?user.id=bf149f246c1644aa85c4c162a2e07984&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:09.475136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.511533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20186a68-eea8-42ea-bfa5-7d290152a298 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.548129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.548787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.549060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.549325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.549915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.551364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:09.562898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:09.564327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:39:09.565312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:09.568317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 402/811] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:09.569092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.608877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31afed-6adc-49cf-ae6a-db4979c92ec9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.628554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.637348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.650374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.652148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.652426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.652703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.664236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.665583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:09.680822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:09.681852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:09.682851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:09.682851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:09.682851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:09.683972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 410/812] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:09.684150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.697401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167b7e2-609d-41eb-9346-936b8e7d1bb8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.740194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.740848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.741127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.741404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] PATH_INFO: `/v3/services/network` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.742003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.743919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:09.754928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:09.756156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:09.757013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 403/813] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:09.757810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.814325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f3cbf-f288-4000-9592-1f430ea0ae92 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.849114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.849757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.850039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.850312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.862285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.863774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.866222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7260cef-e24e-47e8-9eb4-4d944811bb41 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:09.879457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:09.883078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:09.884014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:09.884014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:09.884014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:09.884014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 411/814] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:09.884014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:09.887393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.909365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:09.910227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.910700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.911156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.912141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:09.912441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:09.913804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:09.916981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.924338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 404/815] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:09.924338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.015516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa82a2-9dbf-47f1-9a4b-379aecc4feaa None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.032404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdd9aa-8b2a-46e5-9daa-9251d0fcd40e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.049974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.050592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.050871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.051145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.051842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:10.052171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.053533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.065728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 412/816] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:10.066537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.072871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.074231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.074231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.074231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.074614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] RBAC: Adding query filter params (name=network) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:10.074923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.076306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.085468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 405/817] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:10.085468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.126509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.167298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.180898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b1e2b-fd4a-458d-99a8-74b44282b370 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.185011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4547e-5b40-4886-bc24-14246b02c621 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.215845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.216475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.216752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.217022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.217663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:10.217965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.219320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.231482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.232129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.238743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.238743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.239139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 413/818] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:10.239268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:10.239268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.239485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.239607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.241072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 406/819] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:10.248061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.358437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032e637-b778-43c2-8c72-c1fd9045e470 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.370238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63035963-3723-4cd0-80a2-31a1ca70740b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.377953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.397139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.397843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.398069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.398337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.408056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.408492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.408675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.408956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.409224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.409850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.409957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] RBAC: Adding query filter params (type=network) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:10.410151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.411560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.421929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 407/820] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:10.421929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:10.428336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:10.429497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:10.430519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 414/821] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:10.431121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.442802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.530038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec4bb7-bf8c-4dec-b250-d5716bf3158a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.539193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e44163-f6c4-4102-86db-0e60ea2aa1b6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.565181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.565810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.566117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.566362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.573819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.575295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.578497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.579112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.579392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.579664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.580378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.581814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:10.588597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:10.589645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:10.590726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:10.592848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 408/822] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:10.593636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.612686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 415/823] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:10.613552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.649116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.706138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.736709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a2fff-39cd-4ab5-a54a-c7f6fa190bc1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.783836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.784613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.784891 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.785163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.785981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.787469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.788193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b30a91-28bf-4fe8-8770-b1a9468f064f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.813914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 409/824] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:10.814866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.845982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.847005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.847788 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.850011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.851352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:10.852030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.855266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.885919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 416/825] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:10.887908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.933987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.961589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5c2de-db25-4d04-b0d8-f34b47ccebc5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:11.012139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:11.012876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.013226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.013610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.014633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:11.014999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:11.025309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:11.027709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9ab21-2eb9-409f-aa8f-b5cc65fb4461 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:11.044676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 410/826] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:11.046659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:11.053219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:11.071028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:11.072391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.072895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.073389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.087613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:11.089219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:11.108851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:11.109890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:11.110803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:11.110803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:11.110803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:11.114971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 417/827] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:11.116317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:11.190232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7c537-ee62-4ec9-9ea4-6868620b2115 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:11.237564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:11.239002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.239454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.240359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/3e23eac1e2b04f59bbdac1646da01cbe/roles/4460b00f7dab40cd8eb26f0b606b4101` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.257783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:11.263392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=3e23eac1e2b04f59bbdac1646da01cbe, role_id=4460b00f7dab40cd8eb26f0b606b4101)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:11.265674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:11.269279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36ea11-3bd2-4d27-b1e8-8701562257d5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:11.283492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 411/828] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:39:11 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/3e23eac1e2b04f59bbdac1646da01cbe/roles/4460b00f7dab40cd8eb26f0b606b4101 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:39:11.318590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:11.319276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.319557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.319827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.320615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:11.320966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:11.322371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:11.340874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 418/829] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:11.345870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:11.467901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5234cd5-4efa-4d0a-af55-25a51b2f362b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:11.505158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:11.506132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.506603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.507059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.514620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:11.516408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:11.528983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:11.530067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:11.530965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:11.533477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 412/830] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:11.539090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:11.568985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:11.647919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252a33f-7420-4093-8c5b-b6c4f4feb929 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:11.692986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50d559-4378-4cf8-ab07-b0c477f6656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:11.693603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50d559-4378-4cf8-ab07-b0c477f6656e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.693883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50d559-4378-4cf8-ab07-b0c477f6656e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.694157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50d559-4378-4cf8-ab07-b0c477f6656e None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.694866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50d559-4378-4cf8-ab07-b0c477f6656e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:11.695174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50d559-4378-4cf8-ab07-b0c477f6656e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:11.696750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50d559-4378-4cf8-ab07-b0c477f6656e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:11.716225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 419/831] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:11.721635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:11.779022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:11.786448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f75e3-3844-48d7-9dec-087f89b8083e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.786821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f75e3-3844-48d7-9dec-087f89b8083e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.787002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f75e3-3844-48d7-9dec-087f89b8083e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.788978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 420/832] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:11.797880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a880a-9621-45cc-9736-bc5e7cac29c5 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.798934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a880a-9621-45cc-9736-bc5e7cac29c5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.799661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a880a-9621-45cc-9736-bc5e7cac29c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.820994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f70a880a-9621-45cc-9736-bc5e7cac29c5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:11.822629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f70a880a-9621-45cc-9736-bc5e7cac29c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:11.840626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad70d6-8e56-49d3-9df5-d5be727a59a3 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:11.884764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:11.886255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:11.886729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:11.887191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/c76ff74c7e67426bb2b94a4bb1a2b9ec/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:11.889926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=c76ff74c7e67426bb2b94a4bb1a2b9ec, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:11.892088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:11.911956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 421/833] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:11.920231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 413/834] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:39:11 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/c76ff74c7e67426bb2b94a4bb1a2b9ec/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:39:12.184369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a880a-9621-45cc-9736-bc5e7cac29c5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:12.203530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:12.290122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a880a-9621-45cc-9736-bc5e7cac29c5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:12.327048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:12.327704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:12.328330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:12.328600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:12.329196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:12.330546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:12.339821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:12.340843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:12.341753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 422/835] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Oct 4 08:39:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:12.347109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:12.456443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc874d-d23e-4659-9293-da48f8bd0c2f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:12.470008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:12.493399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d79567-c48d-4cc7-9912-f51c21461486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:12.494024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d79567-c48d-4cc7-9912-f51c21461486 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:12.494345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d79567-c48d-4cc7-9912-f51c21461486 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:12.494623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d79567-c48d-4cc7-9912-f51c21461486 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:12.495263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d79567-c48d-4cc7-9912-f51c21461486 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:12.495565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d79567-c48d-4cc7-9912-f51c21461486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:12.497286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d79567-c48d-4cc7-9912-f51c21461486 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:12.502433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 414/836] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:12.507742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:12.594419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:12.614789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadda6b-28c0-4934-a4cf-1340e8070e49 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:12.649929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf6bf5-24aa-42c1-9c52-6a37042edd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:12.650541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf6bf5-24aa-42c1-9c52-6a37042edd39 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:12.650815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf6bf5-24aa-42c1-9c52-6a37042edd39 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:12.651084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf6bf5-24aa-42c1-9c52-6a37042edd39 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:12.651713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf6bf5-24aa-42c1-9c52-6a37042edd39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:12.652093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf6bf5-24aa-42c1-9c52-6a37042edd39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:12.653447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf6bf5-24aa-42c1-9c52-6a37042edd39 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:12.658515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 423/837] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:13.134117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0e97-8bd3-4796-b7ff-8401a3700421 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:13.134818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0e97-8bd3-4796-b7ff-8401a3700421 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:13.135301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0e97-8bd3-4796-b7ff-8401a3700421 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.137233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 415/838] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:13.143013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d1fc1-d0a1-4195-8c64-e1b0377516ea None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:13.143592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d1fc1-d0a1-4195-8c64-e1b0377516ea None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:13.144118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d1fc1-d0a1-4195-8c64-e1b0377516ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.163470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5c6d1fc1-d0a1-4195-8c64-e1b0377516ea None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:13.165044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5c6d1fc1-d0a1-4195-8c64-e1b0377516ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:13.236513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 424/839] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:13.501879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d20b9-63f8-4693-850e-8da717075534 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:13.502398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d20b9-63f8-4693-850e-8da717075534 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:13.502398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d20b9-63f8-4693-850e-8da717075534 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.503912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 416/840] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:13.505013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d1fc1-d0a1-4195-8c64-e1b0377516ea None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:13.511011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6a71-11f7-491d-bf4f-9a2212c1801a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:13.511290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6a71-11f7-491d-bf4f-9a2212c1801a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:13.511556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6a71-11f7-491d-bf4f-9a2212c1801a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.535500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c2bf6a71-11f7-491d-bf4f-9a2212c1801a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:13.537241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c2bf6a71-11f7-491d-bf4f-9a2212c1801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:13.608797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:13.636449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 417/841] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:13.647702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d1fc1-d0a1-4195-8c64-e1b0377516ea None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:13.682992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:13.683935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:13.684385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:13.684862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.685620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:13.687138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:13.696361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:13.697371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:13.698238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:13.700613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 425/842] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:13.709228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6a71-11f7-491d-bf4f-9a2212c1801a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:13.810288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6a71-11f7-491d-bf4f-9a2212c1801a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:13.818773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:13.845966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:13.846843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:13.847258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:13.847661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.848518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:13.848983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:13.850592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:13.855480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 418/843] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:13.860700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:13.928464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:13.983429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42774a10-71f2-401a-a7dd-4651a343ee1f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.032232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.032899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.033184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.033689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.034332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:14.034638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:14.036904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:14.047959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 426/844] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:14.068776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafebea-b578-4ed7-8440-7597ed6d0d75 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.108216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbea84-a72a-4ee9-8810-7212a70d927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.108906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbea84-a72a-4ee9-8810-7212a70d927d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.109182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbea84-a72a-4ee9-8810-7212a70d927d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.109494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbea84-a72a-4ee9-8810-7212a70d927d None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.122538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbea84-a72a-4ee9-8810-7212a70d927d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d9fbea84-a72a-4ee9-8810-7212a70d927d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:14.127594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:14.128583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:14.129702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 419/845] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:39:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:14.137250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:14.140724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:14.173040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1cf15-22d4-43a7-a0df-1bd56ea4adc1 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.173242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1cf15-22d4-43a7-a0df-1bd56ea4adc1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.173458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1cf15-22d4-43a7-a0df-1bd56ea4adc1 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.175601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 420/846] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:14.181226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd98dd2-0d9c-4cf5-b27d-54207864f47b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.181527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd98dd2-0d9c-4cf5-b27d-54207864f47b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.182569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd98dd2-0d9c-4cf5-b27d-54207864f47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.200882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5fd98dd2-0d9c-4cf5-b27d-54207864f47b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:14.202044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5fd98dd2-0d9c-4cf5-b27d-54207864f47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:14.228548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:14.266979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e7b51-af56-4cc5-8fa3-247ba6b7b078 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.283915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 421/847] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:14.305430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.306254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.306654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.307084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.307876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:14.308352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:14.309823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:14.323370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 427/848] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:39:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:14.324916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd98dd2-0d9c-4cf5-b27d-54207864f47b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:14.431495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd98dd2-0d9c-4cf5-b27d-54207864f47b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.468594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.469298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.469618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.469890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.480865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:14.482543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:14.514541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:14.516271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:14.517158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:14.517158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:14.517158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:14.517158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 422/849] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:14.524117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:14.575057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:14.593643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:14.597555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:14.645586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717cdf-a029-4880-b367-b7ad801299d5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.687749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.688701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.689157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.689264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf023-e714-4837-a9dc-55a0bdc9fde8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.689729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.690826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:14.691319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:14.694078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:14.714669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 428/850] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:14.716827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:14.731486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.732173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.732454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.732722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.744326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:14.750781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:14.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:14.753321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 423/851] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:14.765014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:14.855980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246d105-b042-46d4-a4e7-8ab8631fac53 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.876620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:14.888667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:14.901472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad2225-76e5-4fea-81f4-9c3f7c1d7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.902113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2225-76e5-4fea-81f4-9c3f7c1d7e64 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.902383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2225-76e5-4fea-81f4-9c3f7c1d7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.902970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2225-76e5-4fea-81f4-9c3f7c1d7e64 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.910817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad2225-76e5-4fea-81f4-9c3f7c1d7e64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:14.912303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad2225-76e5-4fea-81f4-9c3f7c1d7e64 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:14.915117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903a335-c199-4764-bdd7-f16a90c4cf4b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-8fad2225-76e5-4fea-81f4-9c3f7c1d7e64 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:14.924556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:14.925552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:14.926655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 429/852] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:39:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:14.927256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdf413-aba1-40cf-ad36-51514c4686c2 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.927256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdf413-aba1-40cf-ad36-51514c4686c2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.928287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdf413-aba1-40cf-ad36-51514c4686c2 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.929087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 430/853] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:14.931224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdf413-aba1-40cf-ad36-51514c4686c2 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:14.960715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecc3da-8165-4367-a127-88a9aa5e7579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:14.961333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecc3da-8165-4367-a127-88a9aa5e7579 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.961620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecc3da-8165-4367-a127-88a9aa5e7579 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.961891 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecc3da-8165-4367-a127-88a9aa5e7579 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.962560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecc3da-8165-4367-a127-88a9aa5e7579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:14.962863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecc3da-8165-4367-a127-88a9aa5e7579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:14.964259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecc3da-8165-4367-a127-88a9aa5e7579 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:14.978560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 424/854] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:14.978560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cd72f-3307-47a2-89de-a63ca21f2a01 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:14.978560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cd72f-3307-47a2-89de-a63ca21f2a01 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:14.978560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cd72f-3307-47a2-89de-a63ca21f2a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:14.996252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b17cd72f-3307-47a2-89de-a63ca21f2a01 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:14.997721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b17cd72f-3307-47a2-89de-a63ca21f2a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:15.055685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdf413-aba1-40cf-ad36-51514c4686c2 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.072308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 425/855] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:15.073052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cd72f-3307-47a2-89de-a63ca21f2a01 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.116570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.118024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.118572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.119015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.120188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:15.120687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.123307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.137554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:15.150248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 431/856] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Oct 4 08:39:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:15.155782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.179157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cd72f-3307-47a2-89de-a63ca21f2a01 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.212673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:15.224087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.225162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.225713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.226153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.238821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.240568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:15.256216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:15.257209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:15.258099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:15.258099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:15.258099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:15.260249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 426/857] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:15.266665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.285568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9bafe-e5b2-4933-9e78-a04710385539 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.341921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.343091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.343960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.344545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.387197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a786b6-9bd6-4625-9ffa-657309588a80 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.406539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:15.422171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.423624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.426970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.427394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.427672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.427986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.428638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:15.428939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.430296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.448038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 427/858] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:15.448038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.472958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:15.540837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 432/859] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 08:39:15.542130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.553293 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd9b5e-58b1-44c1-8b12-bb78fbb8be13 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.590540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.591559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.592374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.593059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.593961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.595969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.615995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 428/860] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:15 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:15.615995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.653284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b0552-90d9-4301-8354-e0b16dd1bdc8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.676666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:15.718402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.719176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.719522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.719905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.730871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.733170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.735554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:15.751439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:15.752541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:15.754147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 433/861] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:15.758730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.764252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e2a1c-c0ff-476b-8236-d04be0e323f5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.804442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.805461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.805995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.806274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.818363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.820352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:15.837802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:15.838997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:15.840147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:15.840147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:15.840147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:15.840147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 429/862] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:15.845792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.880112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec079ad-346b-417b-82d0-d077bebe4690 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:15.927182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:15.927748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:15.928058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:15.928328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:15.929065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:15.929371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:15.930919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:15.947707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:15.953693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 434/863] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:15.960369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:15.963042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160af06-f43d-4416-bccc-940af18d6131 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.005578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.006633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.007182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.007671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.008652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:16.009151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.010836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.023241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:16.025559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 430/864] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:16.031713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.081164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb88ae-11b2-438b-8440-bcbb096ebf49 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.122166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a846484-18da-4444-ad10-3c84d0a374d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.123804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a846484-18da-4444-ad10-3c84d0a374d9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.124330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a846484-18da-4444-ad10-3c84d0a374d9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.125046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a846484-18da-4444-ad10-3c84d0a374d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.142792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c628be-aee0-4bf4-851a-fa09b0f2681d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.182576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.183250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.183972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.184352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.194075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.196371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.210833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a846484-18da-4444-ad10-3c84d0a374d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.212780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a846484-18da-4444-ad10-3c84d0a374d9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:16.213705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:16.214582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:16.215502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:16.219878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 431/865] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:16.220845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.232246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:16.278728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:16.334176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4209f3e-52f4-461d-9849-fd98af725004 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.335217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 435/866] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:15 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:16.336587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb58b4a-787c-4527-9264-9b2075ca37e6 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.336876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb58b4a-787c-4527-9264-9b2075ca37e6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.337179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb58b4a-787c-4527-9264-9b2075ca37e6 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.338965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 436/867] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:16.339159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb58b4a-787c-4527-9264-9b2075ca37e6 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.379206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4081b-b54d-4c9c-90e5-2c133dbfdfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.379906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4081b-b54d-4c9c-90e5-2c133dbfdfae None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.380193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4081b-b54d-4c9c-90e5-2c133dbfdfae None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.380464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4081b-b54d-4c9c-90e5-2c133dbfdfae None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.381155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4081b-b54d-4c9c-90e5-2c133dbfdfae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:16.381467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4081b-b54d-4c9c-90e5-2c133dbfdfae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.382898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4081b-b54d-4c9c-90e5-2c133dbfdfae None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.394833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 432/868] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:16.396112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a185a-e584-4578-a311-0e40bd54ac60 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.396394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a185a-e584-4578-a311-0e40bd54ac60 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.396664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a185a-e584-4578-a311-0e40bd54ac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.414610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ce4a185a-e584-4578-a311-0e40bd54ac60 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:16.415798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ce4a185a-e584-4578-a311-0e40bd54ac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:16.460057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb58b4a-787c-4527-9264-9b2075ca37e6 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.480944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:16.487737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 433/869] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:16 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:16.488590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a185a-e584-4578-a311-0e40bd54ac60 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.505018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125444f6-195e-4750-a96f-320baa316805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.505740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125444f6-195e-4750-a96f-320baa316805 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.506128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125444f6-195e-4750-a96f-320baa316805 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.506465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125444f6-195e-4750-a96f-320baa316805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.518267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125444f6-195e-4750-a96f-320baa316805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.519792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125444f6-195e-4750-a96f-320baa316805 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-125444f6-195e-4750-a96f-320baa316805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:16.535319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:16.536527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:16.537427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:16.537427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:16.537427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:16.541419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 437/870] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:16.542320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125444f6-195e-4750-a96f-320baa316805 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.553766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:16.619153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a185a-e584-4578-a311-0e40bd54ac60 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.656389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.657368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.657853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.658345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.659943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=3e23eac1e2b04f59bbdac1646da01cbe) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:16.660342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.662069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.670236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 434/871] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:16 2022] GET /identity/v3/role_assignments?user.id=3e23eac1e2b04f59bbdac1646da01cbe&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:16.672256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125444f6-195e-4750-a96f-320baa316805 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.714246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.715129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.715553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.715998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.716800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:16.717246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.718759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.733011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 438/872] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:16.739019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.774436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.819951 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:16.876864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34033f-f4e0-40fb-81c2-6ff2e940052f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.879487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:16.918719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.919693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.921004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.921135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef25a32-95fa-4c92-8f77-486c1a01de0f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:16.921607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.929351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.931316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:16.945760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:16.947571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:16.948839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 435/873] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:16.953455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:16.979585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:16.980417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:16.980761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:16.981108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:16.981941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:16.983407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:17.009159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 439/874] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:16 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:39:17.083371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee44a8f-2203-4954-a66d-5877964bea46 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:17.110363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:17.130610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358cb64-a0da-4b2a-9c1f-954901e16488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:17.131220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358cb64-a0da-4b2a-9c1f-954901e16488 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:17.131495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358cb64-a0da-4b2a-9c1f-954901e16488 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:17.131767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358cb64-a0da-4b2a-9c1f-954901e16488 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:17.132501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358cb64-a0da-4b2a-9c1f-954901e16488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:17.132801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358cb64-a0da-4b2a-9c1f-954901e16488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:17.134247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358cb64-a0da-4b2a-9c1f-954901e16488 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:17.148874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 436/875] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:17.154608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:17.265541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fe171-9a96-446b-97c8-5e5757404182 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:17.293159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:17.302708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f1740-cde8-407e-bc5f-66a518c1f692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:17.303330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1740-cde8-407e-bc5f-66a518c1f692 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:17.303609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1740-cde8-407e-bc5f-66a518c1f692 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:17.303903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1740-cde8-407e-bc5f-66a518c1f692 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:17.304647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f1740-cde8-407e-bc5f-66a518c1f692 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:17.304951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f1740-cde8-407e-bc5f-66a518c1f692 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:17.306378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f1740-cde8-407e-bc5f-66a518c1f692 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:17.313159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 440/876] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:17 2022] GET /identity/v3/role_assignments?user.id=c76ff74c7e67426bb2b94a4bb1a2b9ec&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:17.891475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f31ec-1d62-4cba-9224-ddc0cdd214f7 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:17.892056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f31ec-1d62-4cba-9224-ddc0cdd214f7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:17.892056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f31ec-1d62-4cba-9224-ddc0cdd214f7 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:17.893577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 437/877] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:17.898997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cda0a1-711c-4e19-9e90-3b68cb94d953 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:17.899407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cda0a1-711c-4e19-9e90-3b68cb94d953 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:17.899585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cda0a1-711c-4e19-9e90-3b68cb94d953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:17.917686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c1cda0a1-711c-4e19-9e90-3b68cb94d953 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:17.918861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c1cda0a1-711c-4e19-9e90-3b68cb94d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:17.985760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 441/878] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:18.245341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f31ec-1d62-4cba-9224-ddc0cdd214f7 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:18.350197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f31ec-1d62-4cba-9224-ddc0cdd214f7 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:18.391372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:18.392033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:18.392313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:18.392586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:18.393169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:18.394536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:18.403603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:18.404656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:18.405599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 438/879] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Oct 4 08:39:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:18.411277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cda0a1-711c-4e19-9e90-3b68cb94d953 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:18.507649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cda0a1-711c-4e19-9e90-3b68cb94d953 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:18.529686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:18.541247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:18.541851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:18.542173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:18.542438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:18.543066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:18.543364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:18.544737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:18.549641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 442/880] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:18.555273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:18.599385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:18.660003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cb249-1f3f-45c4-9bb3-fe569e0d1fe4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:18.700395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:18.701009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:18.701287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:18.701553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:18.702199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:18.702496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:18.703866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:18.709572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 439/881] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:18.715075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:18.816468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdb5c5-bdfa-4164-8eed-6eb5fbe838ea None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:18.850773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ab445-35ee-490b-bfb8-421066a46fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:18.851394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ab445-35ee-490b-bfb8-421066a46fa4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:18.851642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ab445-35ee-490b-bfb8-421066a46fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:18.851981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ab445-35ee-490b-bfb8-421066a46fa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:18.852624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ab445-35ee-490b-bfb8-421066a46fa4 None admin] RBAC: Adding query filter params (interface=public, service_id=5dc344e0ea35416d8bd6591d5ab254c8, region_id=RegionOne) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:18.852927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ab445-35ee-490b-bfb8-421066a46fa4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:18.854271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ab445-35ee-490b-bfb8-421066a46fa4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:18.859771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 443/882] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:39:18 2022] GET /identity/v3/endpoints?service_id=5dc344e0ea35416d8bd6591d5ab254c8&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:18.865611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:18.977029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba22655-c3e7-4b6a-af58-c082a1da4f4a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:18.985248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:19.016142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e98888-b2db-44b1-8dd6-fb4caae3e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:19.016776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98888-b2db-44b1-8dd6-fb4caae3e497 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.017054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98888-b2db-44b1-8dd6-fb4caae3e497 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.017323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98888-b2db-44b1-8dd6-fb4caae3e497 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.017940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e98888-b2db-44b1-8dd6-fb4caae3e497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:19.019335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e98888-b2db-44b1-8dd6-fb4caae3e497 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:19.026514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 440/883] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:18 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:39:19.122724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e1be-77f7-44e6-ba86-dfcf5ccd7fc6 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.123225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e1be-77f7-44e6-ba86-dfcf5ccd7fc6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.123225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e1be-77f7-44e6-ba86-dfcf5ccd7fc6 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.124923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 444/884] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:19.130749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019cabd-f7e2-422e-82a5-0f2b8dbcb7c4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.131068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019cabd-f7e2-422e-82a5-0f2b8dbcb7c4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.131311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019cabd-f7e2-422e-82a5-0f2b8dbcb7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.151322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6019cabd-f7e2-422e-82a5-0f2b8dbcb7c4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:19.152565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6019cabd-f7e2-422e-82a5-0f2b8dbcb7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:19.228203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 441/885] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:19.328128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f84ac-7df2-4c63-8942-5a4b437d279e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.328757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f84ac-7df2-4c63-8942-5a4b437d279e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.329249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f84ac-7df2-4c63-8942-5a4b437d279e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.331153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 445/886] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:19.336782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96feb07-e795-4974-bb36-67cbbe5d8527 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.337345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96feb07-e795-4974-bb36-67cbbe5d8527 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.337828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96feb07-e795-4974-bb36-67cbbe5d8527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.356411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d96feb07-e795-4974-bb36-67cbbe5d8527 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:19.357941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d96feb07-e795-4974-bb36-67cbbe5d8527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:19.427959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 442/887] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:19.492016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f84ac-7df2-4c63-8942-5a4b437d279e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:19.545965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:19.556071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de540b1-15f3-440f-be32-f83465281e33 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.556354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de540b1-15f3-440f-be32-f83465281e33 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.556621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de540b1-15f3-440f-be32-f83465281e33 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.558220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 443/888] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:19.563297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4068-3a10-43e2-a82b-aedc83e79fd1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.563573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4068-3a10-43e2-a82b-aedc83e79fd1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.563844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd4068-3a10-43e2-a82b-aedc83e79fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.582320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d8cd4068-3a10-43e2-a82b-aedc83e79fd1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:19.583626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d8cd4068-3a10-43e2-a82b-aedc83e79fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:19.598831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f84ac-7df2-4c63-8942-5a4b437d279e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:19.616029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:19.633867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:19.634505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.634782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.635048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.646777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:19.652187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:19.653155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:19.654107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:19.663908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 446/890] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Oct 4 08:39:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:19.663908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:19.664392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 444/890] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:19.711561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4068-3a10-43e2-a82b-aedc83e79fd1 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:19.772615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571ca30-cac5-4a9a-b4e1-072d5c0cd9b2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:19.785503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:19.816631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:19.818143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.818892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.819359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.820354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:19.820916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:19.822461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:19.829486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd4068-3a10-43e2-a82b-aedc83e79fd1 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:19.833937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 447/891] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Oct 4 08:39:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:19.841626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:19.869740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:19.870428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:19.870685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:19.871281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:19.871853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:19.873326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:19.885612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:19.886616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:19.887446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 445/892] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:19.892889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:19.924201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:19.972195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf139-dcd2-4f8f-bf30-2ee3329b5d5d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.005659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01580-e4f5-46fb-a7e2-c6a0d92c4297 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.010011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.010878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.011362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.011835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.023336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.025158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.038085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.038692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.038966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.039236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.039857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:20.040191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:20.040289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:20.041235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:20.042171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:20.042171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:20.042171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:20.042171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 448/893] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:20.042411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.046089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.050926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 446/894] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:20.051979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.074649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:20.149817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:20.163403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf0c52-6cde-4c44-b636-38f38407035d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.164052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c25cb-4802-42d8-9b0f-2e39bf93a494 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.197716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.198325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.198600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.198784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.198897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.199355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.199515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:20.199667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.199774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.199998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.200515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.201245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.201860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:20.211000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:20.212028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:20.212880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 449/895] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:20.213677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.223548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 447/896] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:20.223548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.306582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:20.319871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb12db3-7eed-4161-a79e-c2868cab87be None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.326961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24c10f-f3b2-4cdc-833e-b8f64ccc7676 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.355875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.356540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.356820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.357099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.357760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:20.358063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.359417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.364552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 450/897] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:20.365032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.365332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.366140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.366477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.366812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.367506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:20.367878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.369329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.375414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 448/898] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:20.380285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.398720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:20.470373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a161-e25d-43aa-9c0f-7ed898396dd9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.486376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7895ccd-d16a-4118-b54c-855aeca2fb22 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.508072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.508620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.508906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.509346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.517499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.518927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.519757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.520443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.520727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.520998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.521699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] RBAC: Adding query filter params (interface=public, service_id=7315db5cb1d74072b7a60b32ba2554a7, region_id=RegionOne) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:20.522008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.523366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.528363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 449/899] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/endpoints?service_id=7315db5cb1d74072b7a60b32ba2554a7&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:20.529132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:20.531806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:20.532827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:20.533817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 451/900] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:20.534446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.552704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:20.613494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:20.636617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0275c-ef77-4ec5-a52c-f6abcf072480 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.643834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8112ec-1c93-4b70-89a3-79b6b6fe3fb2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.670107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.670713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.670988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.671258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.671879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:20.672202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.673561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.687919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 450/901] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:20.687919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.690586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.691400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.691780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.692273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.693040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.694556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.702506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 452/902] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:39:20.814548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adce22-1caa-451d-8a37-695f7a72bc92 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:20.854915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f23237-449c-459d-9b20-55e7576f459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:20.855800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f23237-449c-459d-9b20-55e7576f459b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:20.856283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f23237-449c-459d-9b20-55e7576f459b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:20.856741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f23237-449c-459d-9b20-55e7576f459b None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:20.857679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f23237-449c-459d-9b20-55e7576f459b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:20.858172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f23237-449c-459d-9b20-55e7576f459b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:20.859804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f23237-449c-459d-9b20-55e7576f459b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:20.871520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:20.886647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 451/903] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:20.893051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:20.984155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.013351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b749c-c04d-46e6-954c-e6ae47f0241d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:21.062673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:21.063321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.063644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.063944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.139150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:21.140623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:21.217425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f895e7-b74a-4169-97a0-bced867a04f1 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.217965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f895e7-b74a-4169-97a0-bced867a04f1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.217965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f895e7-b74a-4169-97a0-bced867a04f1 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.218579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.223145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 452/904] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:21.224661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254b50c-dbea-43b2-8847-a88a1607fdc6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.224917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254b50c-dbea-43b2-8847-a88a1607fdc6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.225192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254b50c-dbea-43b2-8847-a88a1607fdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.243441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8254b50c-dbea-43b2-8847-a88a1607fdc6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:21.244675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8254b50c-dbea-43b2-8847-a88a1607fdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:21.245343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 453/905] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:20 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:21.250639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:21.340689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 453/906] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:21.386526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bde1c8-547b-4caa-8ecf-5326ca2fd7c1 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:21.422505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:21.423513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.423970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.424384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.434831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:21.436495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:21.451258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:21.452446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:21.453615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:21.453615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:21.453615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:21.455426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 454/907] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:21.460960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254b50c-dbea-43b2-8847-a88a1607fdc6 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:21.567394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254b50c-dbea-43b2-8847-a88a1607fdc6 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:21.596512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.605432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:21.606054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.606333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.606603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.607257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:21.607560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:21.608970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:21.621581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 454/908] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:21.626608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:21.628663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:21.662994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.738099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26757fd9-aac1-4caa-ad0b-3a655e30cde1 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:21.741323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f53b61-2764-4173-8ebc-a5de0279a912 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:21.774412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:21.775056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.775331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.775597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.776213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:21.777946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:21.779627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:21.780411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.780692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.781063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:21.787657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:21.788633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:39:21.789615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:21.790354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:21.790354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:21.792086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 455/909] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Oct 4 08:39:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:21.795020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:21.805779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:21.807502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:21.809141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 455/910] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:21.813608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:21.877553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.904771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.914170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262789-5141-4eb4-9942-7e514b2b7abd None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:21.936723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad81f17-6aa2-4e6d-ac10-4340afbd6f91 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:21.950220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:21.950836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.951114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.951402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.952068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:21.952373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:21.953730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:21.963584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 456/911] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:21.964859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:21.979453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:21.980382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:21.980935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:21.981651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:21.982534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:21.983025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:21.984849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:22.004250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 456/912] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:22.005847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:22.075247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0471a-1b9b-4dd1-8c42-7a2f87cab3c2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:22.109694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:22.110767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:22.111257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:22.111776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:22.112590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:22.113070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:22.114836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:22.117624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2652abd-1e2f-4d16-b9a2-ce2fce17037f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:22.120321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 457/913] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:22.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:22.133647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:22.149854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:22.164070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d18400-e9a2-4416-8225-5d267e286905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:22.165005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18400-e9a2-4416-8225-5d267e286905 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:22.165508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18400-e9a2-4416-8225-5d267e286905 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:22.165971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d18400-e9a2-4416-8225-5d267e286905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:22.166893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18400-e9a2-4416-8225-5d267e286905 None admin] RBAC: Adding query filter params (role.id=96701250d41d426ca366754cd19d50b4, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=3e23eac1e2b04f59bbdac1646da01cbe) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:22.167394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18400-e9a2-4416-8225-5d267e286905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:22.169161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d18400-e9a2-4416-8225-5d267e286905 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:22.175958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 457/914] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:22 2022] GET /identity/v3/role_assignments?user.id=3e23eac1e2b04f59bbdac1646da01cbe&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=96701250d41d426ca366754cd19d50b4 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:22.247230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057039-d405-4536-a856-5dd1a8318f27 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:22.284616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400498-2f4f-4a11-bb07-23c239af5bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:22.285232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400498-2f4f-4a11-bb07-23c239af5bb4 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:22.285511 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400498-2f4f-4a11-bb07-23c239af5bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:22.285821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400498-2f4f-4a11-bb07-23c239af5bb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:22.286497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400498-2f4f-4a11-bb07-23c239af5bb4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:22.287947 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400498-2f4f-4a11-bb07-23c239af5bb4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:22.316113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 458/915] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:39:22.973296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f52c7-fa76-435d-97be-7a6a8956eaf1 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:22.973752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f52c7-fa76-435d-97be-7a6a8956eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:22.973836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f52c7-fa76-435d-97be-7a6a8956eaf1 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:22.975435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 458/916] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:22.980804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee893bb2-d1b5-4aa4-ba0f-4138fc341a23 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:22.981094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee893bb2-d1b5-4aa4-ba0f-4138fc341a23 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:22.981363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee893bb2-d1b5-4aa4-ba0f-4138fc341a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:23.002219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ee893bb2-d1b5-4aa4-ba0f-4138fc341a23 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:23.003437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ee893bb2-d1b5-4aa4-ba0f-4138fc341a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:23.084305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 459/917] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:23.117445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b93a8-c65c-4a3e-b43d-f52ec4b185d6 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:23.117764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b93a8-c65c-4a3e-b43d-f52ec4b185d6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:23.117987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b93a8-c65c-4a3e-b43d-f52ec4b185d6 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:23.119708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 459/918] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:23.125916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895a14-4a91-4458-977e-1e8ac0d0d19d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:23.126232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895a14-4a91-4458-977e-1e8ac0d0d19d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:23.126466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895a14-4a91-4458-977e-1e8ac0d0d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:23.133123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:23.148204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-df895a14-4a91-4458-977e-1e8ac0d0d19d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:23.149389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-df895a14-4a91-4458-977e-1e8ac0d0d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:23.216160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 460/919] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:23.355849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b93a8-c65c-4a3e-b43d-f52ec4b185d6 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:23.463758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b93a8-c65c-4a3e-b43d-f52ec4b185d6 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:23.484017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df895a14-4a91-4458-977e-1e8ac0d0d19d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:23.500311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:23.500930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:23.501208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:23.501477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:23.502140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:23.503512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:23.521642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 460/920] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:23 2022] POST /identity/v3/services => generated 241 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:23.605508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df895a14-4a91-4458-977e-1e8ac0d0d19d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:23.640983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:23.652488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:23.653133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:23.653407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:23.653677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:23.654263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:23.655623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:23.665389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:23.666567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:23.667451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 461/921] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:23.675067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:23.713739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:23.785876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2936ffa-3eb8-4c72-8573-259b91241752 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:23.820035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:23.820651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:23.820927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:23.821196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:23.821832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:23.822137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:23.823623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:23.828984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 461/922] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:23.834776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:23.938149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dffabb-fa0a-4be9-8939-46f6f76499f9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:23.974407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84025a65-1dbd-40b8-be81-361df7016664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:23.975329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84025a65-1dbd-40b8-be81-361df7016664 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:23.975808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84025a65-1dbd-40b8-be81-361df7016664 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:23.976307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84025a65-1dbd-40b8-be81-361df7016664 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:23.977153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84025a65-1dbd-40b8-be81-361df7016664 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:23.977645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84025a65-1dbd-40b8-be81-361df7016664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:23.979195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84025a65-1dbd-40b8-be81-361df7016664 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:23.985207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 462/923] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Oct 4 08:39:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:23.992036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:24.093912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:24.100985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258cddd-02f0-4309-9931-24595bb1e21d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:24.136688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84f1ee-bcd2-4848-9402-6fb38338b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:24.137549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f1ee-bcd2-4848-9402-6fb38338b78f None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:24.137978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f1ee-bcd2-4848-9402-6fb38338b78f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:24.138468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f1ee-bcd2-4848-9402-6fb38338b78f None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:24.139308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84f1ee-bcd2-4848-9402-6fb38338b78f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:24.140850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84f1ee-bcd2-4848-9402-6fb38338b78f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:24.164395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 462/924] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:24.476991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30594971-adc2-4e22-b432-e97b323fd300 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:24.477383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30594971-adc2-4e22-b432-e97b323fd300 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:24.477795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30594971-adc2-4e22-b432-e97b323fd300 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:24.480271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 463/925] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:24.489296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484d4a4-dc82-4e5a-91bc-cbe71a304a8d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:24.489724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484d4a4-dc82-4e5a-91bc-cbe71a304a8d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:24.490097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484d4a4-dc82-4e5a-91bc-cbe71a304a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:24.517273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6484d4a4-dc82-4e5a-91bc-cbe71a304a8d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:24.518975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6484d4a4-dc82-4e5a-91bc-cbe71a304a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:24.595306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa3550-eb9c-42ec-9c18-c98ace728f68 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:24.595830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa3550-eb9c-42ec-9c18-c98ace728f68 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:24.595830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa3550-eb9c-42ec-9c18-c98ace728f68 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:24.597539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 464/926] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:24.598433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 463/927] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:24.602825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d62d-53da-4848-963d-7bf21ac9bffc None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:24.603109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d62d-53da-4848-963d-7bf21ac9bffc None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:24.603381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6d62d-53da-4848-963d-7bf21ac9bffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:24.622296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-75a6d62d-53da-4848-963d-7bf21ac9bffc None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:24.623485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-75a6d62d-53da-4848-963d-7bf21ac9bffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:24.674685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:24.693189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 465/928] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:24.869470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484d4a4-dc82-4e5a-91bc-cbe71a304a8d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:24.956651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:24.980673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484d4a4-dc82-4e5a-91bc-cbe71a304a8d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.016512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.017144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.017409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.017682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.028386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:25.033156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:25.034113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:25.034963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:25.039331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 464/929] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Oct 4 08:39:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:25.040712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6d62d-53da-4848-963d-7bf21ac9bffc None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.067434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.159810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6d62d-53da-4848-963d-7bf21ac9bffc None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.192371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962820e9-bb5c-4d35-94e4-cdfed61fd540 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.212215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.212833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.213111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.213380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.214059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:25.214357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:25.216424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:25.229999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.230640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.230969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.231241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] PATH_INFO: `/v3/services/compute` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.231821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:25.233205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:25.235054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 466/930] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:25.238104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:25.243964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:25.246497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:25.248495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:25.250472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 465/931] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:25.259915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.259915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:25.326094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:25.372800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498087f-8acd-4caf-a3ae-52cc5cd6f544 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.387797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819fa78-96f2-409c-903f-c30a00cdf78c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.414827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.414827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.414827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.414827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.415569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] RBAC: Adding query filter params (name=compute) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:25.415569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:25.420259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:25.422297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 466/932] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:25.427981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.429232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.429908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.430181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.430450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.442406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:25.443980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:25.459561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:25.461327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:25.463442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:25.463442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:25.463442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:25.467918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 467/933] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:25.467918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.499732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:25.539728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f432d-b2f0-4284-8ff0-4010c49b6157 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.576103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.576652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:25.577985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.578329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.578666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.579424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:25.579792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:25.580318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36237a2f-dd53-4053-b90e-dde715ada154 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.581650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:25.587395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 467/934] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:25.621525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.622150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.622426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.622752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.623403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:25.623749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:25.625182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:25.639274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 468/935] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:25.645397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.794637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d46b-dbeb-4cb1-b4d2-94e06d5fe8d5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:25.826744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:25.834032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a20d68-047c-4ee7-a02c-fef20b499fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:25.834960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a20d68-047c-4ee7-a02c-fef20b499fed None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.835381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a20d68-047c-4ee7-a02c-fef20b499fed None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.835831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a20d68-047c-4ee7-a02c-fef20b499fed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.844363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a20d68-047c-4ee7-a02c-fef20b499fed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:25.846117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a20d68-047c-4ee7-a02c-fef20b499fed None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-10a20d68-047c-4ee7-a02c-fef20b499fed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:25.858412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:25.859610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:25.860827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:25.863105 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 468/936] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:25.869117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:25.901982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3a564-cd97-4875-8b0f-d1bf47309ad4 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.902345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3a564-cd97-4875-8b0f-d1bf47309ad4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.902531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3a564-cd97-4875-8b0f-d1bf47309ad4 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.904127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 469/937] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:25.909367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf806be-848b-4b9b-9660-dd113b11d0e8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:25.909646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf806be-848b-4b9b-9660-dd113b11d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:25.909915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf806be-848b-4b9b-9660-dd113b11d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:25.929488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-baf806be-848b-4b9b-9660-dd113b11d0e8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:25.930662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-baf806be-848b-4b9b-9660-dd113b11d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:25.995515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ea91-76c0-4d83-9b76-6597e121687a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:26.023846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 470/938] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:26.037937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:26.038832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.039270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.039737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.040631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:26.041080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:26.042653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:26.062247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 469/939] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Oct 4 08:39:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:26.068268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf806be-848b-4b9b-9660-dd113b11d0e8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.176531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf806be-848b-4b9b-9660-dd113b11d0e8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:26.199489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:26.214500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:26.215143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.215420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.215690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.226768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:26.228270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:26.243671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:26.244711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:26.245643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:26.245643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:26.245643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:26.245643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 471/940] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:26.252373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.273168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:26.364681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b30271-ddf2-4909-bdb2-4aff8434340a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:26.404494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114886c-a6f0-4551-a335-253f743a6935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:26.405505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114886c-a6f0-4551-a335-253f743a6935 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.405981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114886c-a6f0-4551-a335-253f743a6935 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.406463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114886c-a6f0-4551-a335-253f743a6935 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.407302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114886c-a6f0-4551-a335-253f743a6935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:26.407811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114886c-a6f0-4551-a335-253f743a6935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:26.409549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114886c-a6f0-4551-a335-253f743a6935 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:26.422475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 470/941] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:26.428556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.479832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c8c0-8132-4080-a8ff-a0429d115622 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.480163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c8c0-8132-4080-a8ff-a0429d115622 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.480437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c8c0-8132-4080-a8ff-a0429d115622 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.482027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 471/942] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:26.489642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c0040-9204-4993-83bc-9d0173e27386 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.489946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c0040-9204-4993-83bc-9d0173e27386 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.490205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c0040-9204-4993-83bc-9d0173e27386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.509269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-477c0040-9204-4993-83bc-9d0173e27386 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:26.510477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-477c0040-9204-4993-83bc-9d0173e27386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:26.562602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdee15d-6667-421f-8030-727ce09c0a61 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:26.589664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 472/943] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:26.590857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c0040-9204-4993-83bc-9d0173e27386 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.605562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:26.611532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40252114-927e-428f-a798-27befc395905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:26.612715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40252114-927e-428f-a798-27befc395905 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.612715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40252114-927e-428f-a798-27befc395905 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.613107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40252114-927e-428f-a798-27befc395905 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.621720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40252114-927e-428f-a798-27befc395905 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:26.623311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40252114-927e-428f-a798-27befc395905 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-40252114-927e-428f-a798-27befc395905 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:26.636657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:26.637826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:26.638947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:26.640014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 472/944] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:26.649049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40252114-927e-428f-a798-27befc395905 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.695566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:26.718674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c0040-9204-4993-83bc-9d0173e27386 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:26.755627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20732596-6451-427e-928b-37386da586df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:26.756620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732596-6451-427e-928b-37386da586df None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.757073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732596-6451-427e-928b-37386da586df None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.757490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732596-6451-427e-928b-37386da586df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.758253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20732596-6451-427e-928b-37386da586df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:26.759825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20732596-6451-427e-928b-37386da586df None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-20732596-6451-427e-928b-37386da586df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:26.769327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:26.770537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:26.771412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 473/945] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:26.776435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20732596-6451-427e-928b-37386da586df None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.785771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40252114-927e-428f-a798-27befc395905 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:26.827633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:26.828729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.829151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.829552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.831253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:26.831800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:26.835409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:26.868453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 473/946] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:26.869455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:26.870179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.907760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20732596-6451-427e-928b-37386da586df None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:26.944422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:26.945202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.945487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.946081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.946721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:26.947023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:26.948419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:26.958332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 474/947] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:26.959207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:26.990576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:27.001653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb17c9-505b-44d4-a235-fb2397b8419c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:27.041830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:27.042600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.042790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.043037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/3e23eac1e2b04f59bbdac1646da01cbe/roles/96701250d41d426ca366754cd19d50b4` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.045524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=3e23eac1e2b04f59bbdac1646da01cbe, role_id=96701250d41d426ca366754cd19d50b4)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:27.046958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:27.068004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 474/948] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:39:26 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/3e23eac1e2b04f59bbdac1646da01cbe/roles/96701250d41d426ca366754cd19d50b4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:39:27.068967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:27.081978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aa763-04a2-4f0b-8b33-d87fe5d45394 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:27.132863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:27.134358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.134838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.135363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.136263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:27.139088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:27.143481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING keystone.server.flask.common [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 08:39:27.150968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:27.174386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.sql.core [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 08:39:27.174386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 08:39:27.174386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [parameters: {'id': '13e03d6e5ef24de68a50e4043e8bff18', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 08:39:27.174386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103427) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 13e03d6e5ef24de68a50e4043e8bff18.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 13e03d6e5ef24de68a50e4043e8bff18. Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:39:27.184428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result.read() Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 08:39:27.185503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application yield resource Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise self.value Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application session.commit() Oct 04 08:39:27.186692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.session.flush() Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application compat.raise_( Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 08:39:27.187817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 08:39:27.188974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application result.read() Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:39:27.190036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [parameters: {'id': '13e03d6e5ef24de68a50e4043e8bff18', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:27.191431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:27.192565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 13e03d6e5ef24de68a50e4043e8bff18. Oct 04 08:39:27.193701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.197173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 475/949] 10.222.0.44 () {60 vars in 1140 bytes} [Tue Oct 4 08:39:26 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:27.203661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:27.211475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae8cae-d170-48b9-8415-a5c79583d5df None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:27.260919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:27.261908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.262779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.263282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.264313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:27.264945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:27.266760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:27.274107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 475/950] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:27.279724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:27.300457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:27.334832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d18a-ed57-43fd-90e5-46cabff997bd None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:27.377243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:27.378290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.378582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.378857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.387724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:27.390508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:27.401552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed54644-279d-4ab0-8601-9f2b9ca37908 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:27.405356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:27.406258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:27.407350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 476/951] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:27.413403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:27.445927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:27.446463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.447054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.447328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.448555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] RBAC: Adding query filter params (interface=public, service_id=2a794fec64714e9f8a621a2d347dc3d8, region_id=RegionOne) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:27.448868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:27.450770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:27.458642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 476/952] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:39:27 2022] GET /identity/v3/endpoints?service_id=2a794fec64714e9f8a621a2d347dc3d8&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:27.464128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:27.485709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:27.545435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e6a0e-bae7-4316-8d1a-4aa298b0586f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:27.582701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:27.584696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29c267-edbf-45bc-b12f-759a676f33fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:27.585966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c267-edbf-45bc-b12f-759a676f33fb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.586440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c267-edbf-45bc-b12f-759a676f33fb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.586879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c267-edbf-45bc-b12f-759a676f33fb None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.587875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29c267-edbf-45bc-b12f-759a676f33fb None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:27.588379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29c267-edbf-45bc-b12f-759a676f33fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:27.590358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29c267-edbf-45bc-b12f-759a676f33fb None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:27.606196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bed92-c630-40cc-9ecb-3b34e61dee38 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:27.614623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 477/953] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:27.653883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a0592-1b0d-4373-9047-02abddfa41c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:27.654523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a0592-1b0d-4373-9047-02abddfa41c6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.654802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a0592-1b0d-4373-9047-02abddfa41c6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.655073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a0592-1b0d-4373-9047-02abddfa41c6 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.655701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a0592-1b0d-4373-9047-02abddfa41c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:27.657165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a0592-1b0d-4373-9047-02abddfa41c6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:27.664705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 477/954] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:27 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:39:27.947015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed18e1-2b41-488e-a1b8-9967d3d8d655 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.947554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed18e1-2b41-488e-a1b8-9967d3d8d655 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.947554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed18e1-2b41-488e-a1b8-9967d3d8d655 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.949161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 478/955] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:27.954881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890b8ce-514b-452b-9e63-e926ecaa1055 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.955166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890b8ce-514b-452b-9e63-e926ecaa1055 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.955441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890b8ce-514b-452b-9e63-e926ecaa1055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.975139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a890b8ce-514b-452b-9e63-e926ecaa1055 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:27.976374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a890b8ce-514b-452b-9e63-e926ecaa1055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:28.056558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 478/956] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:28.327828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed18e1-2b41-488e-a1b8-9967d3d8d655 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:28.445010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed18e1-2b41-488e-a1b8-9967d3d8d655 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:28.459721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:28.483055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39243651-b17e-40c9-bb55-3e3ab614b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:28.483667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243651-b17e-40c9-bb55-3e3ab614b57f None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:28.483977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243651-b17e-40c9-bb55-3e3ab614b57f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:28.484309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243651-b17e-40c9-bb55-3e3ab614b57f None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:28.484988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39243651-b17e-40c9-bb55-3e3ab614b57f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:28.486348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39243651-b17e-40c9-bb55-3e3ab614b57f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:28.502710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 479/957] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:28 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:29.411582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757f8f-3896-44f4-b33d-aa0e88182532 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:29.412374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757f8f-3896-44f4-b33d-aa0e88182532 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:29.412374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757f8f-3896-44f4-b33d-aa0e88182532 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:29.414338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 479/958] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:29.420208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ff6e-6553-43ef-9a4f-e608581e94f8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:29.420811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ff6e-6553-43ef-9a4f-e608581e94f8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:29.420973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ff6e-6553-43ef-9a4f-e608581e94f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:29.441589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-79a3ff6e-6553-43ef-9a4f-e608581e94f8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:29.443279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-79a3ff6e-6553-43ef-9a4f-e608581e94f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:29.528028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 480/959] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:29.791110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757f8f-3896-44f4-b33d-aa0e88182532 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:29.869449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:29.891280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e24d6-01a2-4b34-b406-b7f3dd55d27d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:29.891554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e24d6-01a2-4b34-b406-b7f3dd55d27d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:29.891831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e24d6-01a2-4b34-b406-b7f3dd55d27d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:29.893361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 481/960] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:29.898974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23241b9-0563-4273-9132-bcdff5821027 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:29.899250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23241b9-0563-4273-9132-bcdff5821027 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:29.899518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23241b9-0563-4273-9132-bcdff5821027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:29.902033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757f8f-3896-44f4-b33d-aa0e88182532 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:29.920231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f23241b9-0563-4273-9132-bcdff5821027 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:29.921552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f23241b9-0563-4273-9132-bcdff5821027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:29.943239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:29.945047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:29.945509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:29.946214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:29.959587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:29.966000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:29.967006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:29.967914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:29.974206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 480/961] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Oct 4 08:39:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:29.976547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.007972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 482/962] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:30.087481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b66edd-3359-4f1e-b2a5-64d14f3d6ba9 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.123473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.124475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.124967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.125461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.126328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:30.126847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.128507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.130100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.144088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 481/963] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Oct 4 08:39:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:30.145846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23241b9-0563-4273-9132-bcdff5821027 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.177117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.250016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23241b9-0563-4273-9132-bcdff5821027 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.285028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.285693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.285974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.286246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.290961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.297292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.298815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:30.316862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:30.317852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:30.318764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:30.318764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:30.318764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:30.319960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 483/964] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:30.324573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.403128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8855e-f849-47a5-a90b-c934fd165c03 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.423255 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.447839 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7af22-04b6-40e1-856c-179093ee5f75 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.449941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.451169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.451664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.452166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.453010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.454718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.457795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING keystone.server.flask.common [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 08:39:30.472102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.sql.core [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 08:39:30.472102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 08:39:30.472102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [parameters: {'user_id': '9244679c25b349b88c7bd1ddba1e2ccc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 08:39:30.472102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103426) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 08:39:30.485147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.485768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.486045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.486320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.486969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:30.487273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.488709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application result.read() Oct 04 08:39:30.492585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:39:30.493477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application yield resource Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise self.value Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application session.commit() Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 08:39:30.494532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self.session.flush() Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application compat.raise_( Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 08:39:30.495451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:39:30.496707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application result.read() Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:39:30.497563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [parameters: {'user_id': '9244679c25b349b88c7bd1ddba1e2ccc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:30.498435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 08:39:30.501132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 08:39:30.503340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 08:39:30.503340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 08:39:30.503340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 08:39:30.503340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 08:39:30.503340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 08:39:30.503340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:30.510781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 484/966] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:30.510781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.511917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 482/966] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Oct 4 08:39:30 2022] POST /identity/v3/users => generated 160 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:30.512815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.578035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.613585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0dbf7-cdad-4654-b015-5b583c52b26b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.627631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b04404-b378-4a65-ac2c-40afeb6bc27a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.651744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139e919-cb53-457b-a360-ce81ec19d6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.652487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e919-cb53-457b-a360-ce81ec19d6ae None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.652748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e919-cb53-457b-a360-ce81ec19d6ae None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.652954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e919-cb53-457b-a360-ce81ec19d6ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.660947 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139e919-cb53-457b-a360-ce81ec19d6ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.662390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139e919-cb53-457b-a360-ce81ec19d6ae None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.665415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.665537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.666179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.666456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.666723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.674466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-3139e919-cb53-457b-a360-ce81ec19d6ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:30.676198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:30.677186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:39:30.678173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:30.678753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.686042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 485/967] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:30.687575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad5c8-0379-4e4b-b58f-e70f1facb08e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.688079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad5c8-0379-4e4b-b58f-e70f1facb08e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:30.688516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:30.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 483/968] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:30.690414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.691606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad5c8-0379-4e4b-b58f-e70f1facb08e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.693040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 486/969] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:30.698772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ad5c8-0379-4e4b-b58f-e70f1facb08e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.802560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ad5c8-0379-4e4b-b58f-e70f1facb08e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.804861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf167c-572d-4252-9f03-f24521442df8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.821382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.837159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5591b-fe40-419a-a34f-16a7fec41dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.837767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5591b-fe40-419a-a34f-16a7fec41dd0 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.838043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5591b-fe40-419a-a34f-16a7fec41dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.838312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5591b-fe40-419a-a34f-16a7fec41dd0 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.838860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c817e5-0dcd-4f79-a43f-7127c7b32fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.839012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5591b-fe40-419a-a34f-16a7fec41dd0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:30.839308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5591b-fe40-419a-a34f-16a7fec41dd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.839479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c817e5-0dcd-4f79-a43f-7127c7b32fe6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.839753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c817e5-0dcd-4f79-a43f-7127c7b32fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.840068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c817e5-0dcd-4f79-a43f-7127c7b32fe6 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.840737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5591b-fe40-419a-a34f-16a7fec41dd0 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.840831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c817e5-0dcd-4f79-a43f-7127c7b32fe6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:30.841090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c817e5-0dcd-4f79-a43f-7127c7b32fe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.842497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c817e5-0dcd-4f79-a43f-7127c7b32fe6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.862587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 484/970] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:30.864604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc53e86-ddcc-41b3-aecd-9b673f9b6daf None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.864604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc53e86-ddcc-41b3-aecd-9b673f9b6daf None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.864604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc53e86-ddcc-41b3-aecd-9b673f9b6daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.877366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 487/971] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:30.877582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd532e-23d1-4e69-a779-f21229990ef1 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.877866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd532e-23d1-4e69-a779-f21229990ef1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.878139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd532e-23d1-4e69-a779-f21229990ef1 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.879604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 488/972] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:30.892855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd532e-23d1-4e69-a779-f21229990ef1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.893108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9fc53e86-ddcc-41b3-aecd-9b673f9b6daf None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:30.893108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9fc53e86-ddcc-41b3-aecd-9b673f9b6daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:30.958955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.987918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 485/973] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:30.987918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74717841-b231-419f-b346-d117871c8de0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.987918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74717841-b231-419f-b346-d117871c8de0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.987918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74717841-b231-419f-b346-d117871c8de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.007813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-74717841-b231-419f-b346-d117871c8de0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:31.009210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-74717841-b231-419f-b346-d117871c8de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:31.023447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd532e-23d1-4e69-a779-f21229990ef1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.069653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.070316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.070598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.070867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.100199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 486/974] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:31.146552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.148035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.208203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.256039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 489/975] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:31.257527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74717841-b231-419f-b346-d117871c8de0 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.268928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.293312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.360724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74717841-b231-419f-b346-d117871c8de0 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.373828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975f9d9-d465-4746-9808-6a57a69b2fc1 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.397301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.397932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.398204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.398469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.408000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.408350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.408652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.408925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.409192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.409765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.409873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.411154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:31.420001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:31.421066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:31.421948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 490/976] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:31.422874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:31.423782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:31.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:31.425617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:31.425617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:31.425617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:31.425617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 487/977] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:31.430165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.451847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.517449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.526341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae4eb6-11bf-496c-a782-65df800ddd38 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.530483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a12adb-330f-4de3-ab7e-959c3091069b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.560692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.561318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.561579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.561852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] PATH_INFO: `/v3/services/compute` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.562498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.563953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.565055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.565654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.565930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.566196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.566872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:31.567167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.568542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:31.572905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:31.574051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:31.576960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:31.577767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 488/978] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:31.577845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 491/979] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:31.577920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.579374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.665581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.673823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a7ae-7820-47fa-84b5-7c7182a5c6d0 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.681841 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f6855-edd9-4a44-91a5-b185db5f1cea None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.706535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.707135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.707468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.707736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.708404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:31.708706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.710066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.717006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.717606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.717885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.718157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.718786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:31.719085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.720487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.726166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 489/980] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:31.726166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.726422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 492/981] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:31.727194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.739799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.827537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2284b-f12c-4051-9ff8-7f73f3d8d01a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.831109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c67f4-2025-4b78-b8e9-e7b5d1cd0f53 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.861538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.862141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.862417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.862685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.863304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] RBAC: Adding query filter params (name=compute) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:31.863660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.864807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:31.865103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.865427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.865699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.865965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.870104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 490/982] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:31.870859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.872816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:31.874383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:31.886298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:31.887244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:31.888389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 493/983] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:31.889083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:31.900726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.956003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:31.981124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421ee7d-a562-4f43-92c2-a413f4ebc427 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:31.999514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f280fb3-3cdd-4905-b8f8-06c1e1a2209e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:32.013782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:32.014380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:32.014655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:32.014924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.015575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:32.016992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:32.033066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:32.033665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:32.033944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:32.034216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.034835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] RBAC: Adding query filter params (type=compute) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:32.035135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:32.036535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:32.036886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 491/984] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:32.037705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:32.041644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 494/985] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:32.047706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:32.140883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:32.165557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56bc6-8832-42ac-9f9a-020b73857805 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:32.167731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc38ce5-7196-444f-bedb-f719f1749f59 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:32.197407 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:32.206977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:32.207362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:32.207626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:32.207914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:32.208037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:32.208208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.208315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:32.208520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.208905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] RBAC: Adding query filter params (interface=public, service_id=866a00f479a04b68ae4363f7434269f5, region_id=RegionOne) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:32.209201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:32.209296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:32.209505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:32.210637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:32.210912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:32.217159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 495/986] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:39:32 2022] GET /identity/v3/endpoints?service_id=866a00f479a04b68ae4363f7434269f5&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:32.222999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:32.226526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 492/987] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:32.232649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:32.336204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea833-9eb7-4637-910f-c3c214738454 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:32.336713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d33ad-201f-4260-bc6f-51eab65c5fd2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:32.370783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ed73a-4819-47a4-9a4c-ccbd06ecd244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:32.371801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ed73a-4819-47a4-9a4c-ccbd06ecd244 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:32.372194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ed73a-4819-47a4-9a4c-ccbd06ecd244 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:32.372474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ed73a-4819-47a4-9a4c-ccbd06ecd244 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.372648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b27565-4f28-4de4-91f0-3b428a15a86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:32.373099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ed73a-4819-47a4-9a4c-ccbd06ecd244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:32.373269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27565-4f28-4de4-91f0-3b428a15a86d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:32.373531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27565-4f28-4de4-91f0-3b428a15a86d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:32.373796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27565-4f28-4de4-91f0-3b428a15a86d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.374461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ed73a-4819-47a4-9a4c-ccbd06ecd244 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:32.374568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b27565-4f28-4de4-91f0-3b428a15a86d None admin] RBAC: Adding query filter params (role.id=96701250d41d426ca366754cd19d50b4, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=3e23eac1e2b04f59bbdac1646da01cbe) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:32.374813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b27565-4f28-4de4-91f0-3b428a15a86d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:32.376421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b27565-4f28-4de4-91f0-3b428a15a86d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:32.387918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 496/988] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:32 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:39:32.388148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 493/989] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:32 2022] GET /identity/v3/role_assignments?user.id=3e23eac1e2b04f59bbdac1646da01cbe&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=96701250d41d426ca366754cd19d50b4 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:34.246744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99939c-9b6d-4836-bcea-3d199525a2a3 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.247261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99939c-9b6d-4836-bcea-3d199525a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.247261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99939c-9b6d-4836-bcea-3d199525a2a3 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.248506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 497/990] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:34.254017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669f71-9732-4fe0-9038-446389702309 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.254424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669f71-9732-4fe0-9038-446389702309 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.254633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669f71-9732-4fe0-9038-446389702309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.273501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6f669f71-9732-4fe0-9038-446389702309 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:34.274867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6f669f71-9732-4fe0-9038-446389702309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:34.328931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:34.350746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 494/991] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:34.574716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa27a0a-9054-4301-af2a-3f3140ce501b None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.575011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa27a0a-9054-4301-af2a-3f3140ce501b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.575599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa27a0a-9054-4301-af2a-3f3140ce501b None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.578005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 498/992] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:34.581894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b037f8-44a4-4a85-a5e3-3027d3dad5c7 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.582253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b037f8-44a4-4a85-a5e3-3027d3dad5c7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.582566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b037f8-44a4-4a85-a5e3-3027d3dad5c7 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.583616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c22e-9ab9-41e9-aedf-a08301e099ce None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.584144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c22e-9ab9-41e9-aedf-a08301e099ce None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.584280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 495/993] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:34.584359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c22e-9ab9-41e9-aedf-a08301e099ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.589076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697f9f1-2919-4c7a-a553-eb0c2cd8042a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.589360 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697f9f1-2919-4c7a-a553-eb0c2cd8042a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.589631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697f9f1-2919-4c7a-a553-eb0c2cd8042a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.591529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:34.607759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d556c22e-9ab9-41e9-aedf-a08301e099ce None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:34.608083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6697f9f1-2919-4c7a-a553-eb0c2cd8042a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:34.609029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6697f9f1-2919-4c7a-a553-eb0c2cd8042a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:34.609156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d556c22e-9ab9-41e9-aedf-a08301e099ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:34.676996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 496/994] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:34.677821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697f9f1-2919-4c7a-a553-eb0c2cd8042a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:34.683194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 499/995] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:34.785112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697f9f1-2919-4c7a-a553-eb0c2cd8042a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:34.798105 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:34.820911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:34.821541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.821818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.822083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.822663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:34.824049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:34.832987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:34.834042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:34.835309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 497/996] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:34.839978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556c22e-9ab9-41e9-aedf-a08301e099ce None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:34.940815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556c22e-9ab9-41e9-aedf-a08301e099ce None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:34.944049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:34.972662 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:34.978026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:34.978632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.978911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.979178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.979808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:34.980140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:34.981494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:34.986664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 500/997] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:34.987483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.053795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7d41-931c-4423-8100-050d301f16c2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.083720 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e9df-7392-4ca8-96f6-cec3dd2ca91d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.087176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.087870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.088184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.088442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] PATH_INFO: `/v3/services/image` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.089015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.090419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:35.099377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:35.100438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:35.101286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 498/998] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Oct 4 08:39:34 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:35.102147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.121113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.122085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.122357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.122622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] PATH_INFO: `/v3/services/compute` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.123183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.124558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:35.133829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:35.134966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:35.135859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 501/999] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:35.136757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.145838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:35.200280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:35.206133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e82b3-fb3d-4c9b-be40-e3bc96ea5813 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.241441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550267b-8655-4df0-b882-761529c3a139 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.241722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.242331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.242619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.242866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.243496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:35.243798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.245201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.250325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 499/1000] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:35.251091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.277881 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.278557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.278820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.279090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.279758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] RBAC: Adding query filter params (name=image) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:35.280090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.281492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.290535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 502/1001] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:35.290535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.347779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f81ce3-2f76-4446-b435-558d179066a4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.350083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:35.381728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.382334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.382613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.382884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.383514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:35.383814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.385210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.388019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19380c5-d161-4895-8041-18cc7e23a8f9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.390043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 500/1002] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:35.390834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.412361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:35.423559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.424192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.424467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.424730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.425419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] RBAC: Adding query filter params (interface=admin, service_id=2a794fec64714e9f8a621a2d347dc3d8, region_id=RegionOne) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:35.425718 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.427065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.432009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 503/1003] 10.222.0.44 () {56 vars in 1249 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/endpoints?service_id=2a794fec64714e9f8a621a2d347dc3d8&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:35.432769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.494234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd95e7-eeaa-428a-9134-cf24586db6d9 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.527717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.528336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.528611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.528876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.529571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] RBAC: Adding query filter params (type=image) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:35.529867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.529988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42504a-d71e-4923-a6b5-c417ecc504aa None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.531266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.536488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 501/1004] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:35.537098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.570182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.570858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.571140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.571412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.572095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] RBAC: Adding query filter params (type=compute) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:35.572804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.573893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.585094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 504/1005] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:35.586124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.586641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:35.649217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:35.663998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33837f2-5a53-40cd-8dec-131dcc9b6dd1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.704383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0aa28-211e-4bcf-b5fb-ca12648ea88d None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:35.705130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668a4eb-a952-4d88-9e40-b169fbfc5821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.705974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668a4eb-a952-4d88-9e40-b169fbfc5821 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.706462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668a4eb-a952-4d88-9e40-b169fbfc5821 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.706728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668a4eb-a952-4d88-9e40-b169fbfc5821 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.707379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668a4eb-a952-4d88-9e40-b169fbfc5821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.708839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668a4eb-a952-4d88-9e40-b169fbfc5821 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.716640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 502/1006] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:39:35.747523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb06f5-67f2-4bd4-a29f-6ae2baa20155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:35.748297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb06f5-67f2-4bd4-a29f-6ae2baa20155 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.748583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb06f5-67f2-4bd4-a29f-6ae2baa20155 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.748862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb06f5-67f2-4bd4-a29f-6ae2baa20155 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.749567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb06f5-67f2-4bd4-a29f-6ae2baa20155 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:35.751082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb06f5-67f2-4bd4-a29f-6ae2baa20155 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:35.780021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 505/1007] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:37.739827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f03bc-1c50-4c71-b06a-39568262dbb3 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:37.740386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f03bc-1c50-4c71-b06a-39568262dbb3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:37.740386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f03bc-1c50-4c71-b06a-39568262dbb3 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:37.741812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 503/1008] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:37.747534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11600cc-d478-4c7b-91a1-291c69076831 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:37.747984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11600cc-d478-4c7b-91a1-291c69076831 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:37.748160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11600cc-d478-4c7b-91a1-291c69076831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:37.767880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c11600cc-d478-4c7b-91a1-291c69076831 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:37.769673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c11600cc-d478-4c7b-91a1-291c69076831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:37.859251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 506/1009] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:37 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:37.893902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086b0e6-a583-413e-b9df-3fa53b1904d7 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:37.894248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086b0e6-a583-413e-b9df-3fa53b1904d7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:37.894447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086b0e6-a583-413e-b9df-3fa53b1904d7 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:37.896113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 504/1010] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:37.902081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96e021-2e4b-4004-a304-f57f66da069e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:37.902385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96e021-2e4b-4004-a304-f57f66da069e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:37.902648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96e021-2e4b-4004-a304-f57f66da069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:37.922203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9a96e021-2e4b-4004-a304-f57f66da069e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:37.923409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9a96e021-2e4b-4004-a304-f57f66da069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:37.954876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:37.999062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 507/1011] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:37 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:38.020927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b8e0-16d6-4f0f-b416-c1d7f4d65b75 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.021197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b8e0-16d6-4f0f-b416-c1d7f4d65b75 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.021468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788b8e0-16d6-4f0f-b416-c1d7f4d65b75 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.022958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 505/1012] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:38.027868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28899da9-a315-4faa-ae50-c3c889644491 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.028234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28899da9-a315-4faa-ae50-c3c889644491 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.028606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28899da9-a315-4faa-ae50-c3c889644491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.047621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-28899da9-a315-4faa-ae50-c3c889644491 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:38.049180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-28899da9-a315-4faa-ae50-c3c889644491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:38.118391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 508/1013] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:38 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:38.122900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b8e0-16d6-4f0f-b416-c1d7f4d65b75 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.222239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.230709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788b8e0-16d6-4f0f-b416-c1d7f4d65b75 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.264495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433444af-765a-42f0-825e-982f94efb159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.265038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433444af-765a-42f0-825e-982f94efb159 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.265309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433444af-765a-42f0-825e-982f94efb159 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.265573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433444af-765a-42f0-825e-982f94efb159 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.266208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433444af-765a-42f0-825e-982f94efb159 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.267563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433444af-765a-42f0-825e-982f94efb159 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.271330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28899da9-a315-4faa-ae50-c3c889644491 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.291910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 506/1014] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:38 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:38.373892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.382600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28899da9-a315-4faa-ae50-c3c889644491 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.389455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433444af-765a-42f0-825e-982f94efb159 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.433128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.433991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.434133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.434316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.434906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.436327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:38.450339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:38.451689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:38.452596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 509/1015] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Oct 4 08:39:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:38.460032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.508290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433444af-765a-42f0-825e-982f94efb159 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.548199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.548839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.549109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.549378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.559516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:38.564802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:38.565873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:38.566873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 507/1016] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Oct 4 08:39:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:38.572538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.584252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744c952-1ee0-4f59-83e1-964f597eafa4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.591806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.623653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.624283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.624744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.624851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.625510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:38.625751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.627169 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.632450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 510/1017] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:38.637939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.658270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.681329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39578d9b-2edc-4da4-a38f-406ace95788b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.715415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.716053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.716331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.716599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.717259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:38.717556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.718917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.731770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 508/1018] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Oct 4 08:39:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:38.732698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.748490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0fbee-845e-4908-a76b-c7b64cce5950 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.783426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.784110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.784394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.784667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.786956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:38.786956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.786956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.791994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 511/1019] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Oct 4 08:39:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:38.797787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.816764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.842480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d272722-1e1f-4ad0-bf60-1a6ba0f21321 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.876213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.876847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.877151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.877421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.885815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.889106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.890643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:38.904528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:38.905531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:38.911416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:38.911416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:38.911416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:38.911591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc84e7-42bb-4a28-9f3a-b4c0b35aad62 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:38.911684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 509/1020] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:38.911684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:38.951254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:38.951873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:38.952171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:38.952439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:38.953113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.954463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.973903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 512/1021] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:39:39.033682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be13707-f3c8-4637-a3d3-773fe568d63c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:39.067265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:39.081941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:39.082598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.082870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.083136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.083775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:39.084111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:39.085591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:39.103932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 510/1022] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:39.106181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:39.220546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a494e8-5492-43c1-87f4-10ed64106b03 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:39.256842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:39.257469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.257743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.258008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.266040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:39.267830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:39.279581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:39.280602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:39.281615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 513/1023] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:39:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:39.288166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:39.389725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e38ea-daec-4ce7-b813-161d0a1ee805 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:39.423523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:39.424153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.424424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.424689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.425404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:39.425767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:39.427503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:39.450052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 511/1024] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:39:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:39.451444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:39.454901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:39.559218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96594c59-f69e-495b-80e5-047c5a577d35 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:39.594147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:39.594784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.595060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.595325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.662174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:39.663528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:39.665340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:39.767166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 514/1025] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:39.773070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:39.819300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:39.875966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5e18-6a0d-4334-9a67-b45ba5190352 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:39.912084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:39.912776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.912998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.913265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.923315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:39.924780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:39.939233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:39.940460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:39.941379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:39.941379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:39.941379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:39.941379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 512/1026] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:39.947778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:40.051927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48188c7e-2eba-403d-a416-94ec7a7e1e4e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:40.053577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:40.088304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:40.088904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.089176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.089441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.090071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:40.090363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:40.091709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:40.103850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 515/1027] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:40.109274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:40.208523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:40.212372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6282e0-0bcf-49aa-9e71-0733467d2464 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:40.248246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:40.248908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.249192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.249443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.256365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:40.257804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:40.269092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:40.270047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:40.270994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 513/1028] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:40.276261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:40.382810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d883d0-077b-4756-a555-e7fba67f47c7 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:40.417801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed56cd-6835-49de-a7f4-6662ac39263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:40.418464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed56cd-6835-49de-a7f4-6662ac39263f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.418738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed56cd-6835-49de-a7f4-6662ac39263f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.418998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed56cd-6835-49de-a7f4-6662ac39263f None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.419675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed56cd-6835-49de-a7f4-6662ac39263f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:40.419998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed56cd-6835-49de-a7f4-6662ac39263f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:40.421395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed56cd-6835-49de-a7f4-6662ac39263f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:40.433729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 516/1029] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:40.439825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:40.465127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee961-7a34-41c3-a265-ce5684670da4 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.465398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee961-7a34-41c3-a265-ce5684670da4 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.465603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee961-7a34-41c3-a265-ce5684670da4 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.467235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 517/1030] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:40.473465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ebc6f-4f1b-4488-a54e-5b3828b53d3a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.473745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ebc6f-4f1b-4488-a54e-5b3828b53d3a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.474012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ebc6f-4f1b-4488-a54e-5b3828b53d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.494470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1f4ebc6f-4f1b-4488-a54e-5b3828b53d3a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:40.495644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1f4ebc6f-4f1b-4488-a54e-5b3828b53d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:40.514804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:40.551281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47df7c-bb67-4837-9367-1dbf858eb4ba None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:40.576953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 518/1031] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:40.589528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:40.590206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.590481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.590743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.591462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] RBAC: Adding query filter params (role.id=96701250d41d426ca366754cd19d50b4, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=bf149f246c1644aa85c4c162a2e07984) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:40.591761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:40.593207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:40.599605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 514/1032] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:40 2022] GET /identity/v3/role_assignments?user.id=bf149f246c1644aa85c4c162a2e07984&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=96701250d41d426ca366754cd19d50b4 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:40.845773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ebc6f-4f1b-4488-a54e-5b3828b53d3a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:40.957767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ebc6f-4f1b-4488-a54e-5b3828b53d3a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:40.997039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:40.997697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.998045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.998320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] PATH_INFO: `/v3/services/image` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.998952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.000392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:41.009890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:41.010943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:41.011968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:41.015952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 519/1033] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Oct 4 08:39:40 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:41.017603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:41.041616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.134522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d033b36-cd02-4e73-b9c1-ed77306ae00f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:41.169930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effc276-facc-4510-b6fc-202b4af0f0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:41.170580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effc276-facc-4510-b6fc-202b4af0f0b8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.170867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effc276-facc-4510-b6fc-202b4af0f0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.171152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effc276-facc-4510-b6fc-202b4af0f0b8 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.171809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effc276-facc-4510-b6fc-202b4af0f0b8 None admin] RBAC: Adding query filter params (name=image) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:41.172157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effc276-facc-4510-b6fc-202b4af0f0b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.173559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effc276-facc-4510-b6fc-202b4af0f0b8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.178634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 515/1034] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:41.184639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:41.191858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512b023-2a61-4f48-bc92-97c0f8b12977 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.192211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512b023-2a61-4f48-bc92-97c0f8b12977 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.192484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512b023-2a61-4f48-bc92-97c0f8b12977 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.193849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 516/1035] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:41.199046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe1065-89ab-4d4b-8619-c701eaeb5859 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.199326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe1065-89ab-4d4b-8619-c701eaeb5859 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.199594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe1065-89ab-4d4b-8619-c701eaeb5859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.204798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.219281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-43fe1065-89ab-4d4b-8619-c701eaeb5859 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:41.220640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-43fe1065-89ab-4d4b-8619-c701eaeb5859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:41.289409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 517/1036] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:41.294146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fd51b-e6ce-4fb1-bc9b-bc4cabe74b3a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:41.327957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:41.328580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.328847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.329122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.329749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] RBAC: Adding query filter params (type=image) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:41.330050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.331404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.336565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 520/1037] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:41.342521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe1065-89ab-4d4b-8619-c701eaeb5859 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:41.358455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.444822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe1065-89ab-4d4b-8619-c701eaeb5859 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:41.481000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:41.481629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.481903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.482173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.482892 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] RBAC: Adding query filter params (interface=public, service_id=178477b57780450b86ab760c9fdfd6ee, region_id=RegionOne) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:41.483199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.484604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.490023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 518/1038] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:39:41 2022] GET /identity/v3/endpoints?service_id=178477b57780450b86ab760c9fdfd6ee&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:41.495782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:41.552491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:41.581047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.603341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305e137-f2c4-4171-ad93-5321757a7183 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:41.638613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:41.639254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.639530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.639802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.640451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.641413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.641876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.648644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 521/1039] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:39:41 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:39:41.679016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ef409-6a8d-4638-9c95-a0cad8bc6798 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:41.718439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979f2e5-104e-4185-be8a-8f2d48bc2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:41.719011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f2e5-104e-4185-be8a-8f2d48bc2b19 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.719288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f2e5-104e-4185-be8a-8f2d48bc2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.719556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f2e5-104e-4185-be8a-8f2d48bc2b19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.720750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979f2e5-104e-4185-be8a-8f2d48bc2b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.722203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979f2e5-104e-4185-be8a-8f2d48bc2b19 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.724194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 519/1040] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:41.730560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:41.847398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2e39a-9929-42f4-aed7-5ed94ff9ab5f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:41.884061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b409f6-df20-4f54-9ee5-3295391764fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:41.884674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b409f6-df20-4f54-9ee5-3295391764fe None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.884955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b409f6-df20-4f54-9ee5-3295391764fe None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.885227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b409f6-df20-4f54-9ee5-3295391764fe None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.885898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b409f6-df20-4f54-9ee5-3295391764fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.887327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b409f6-df20-4f54-9ee5-3295391764fe None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.904790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.919100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 522/1041] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:39:41 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:42.724477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a7006-5a73-4908-8dc5-69641524359e None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:42.725135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a7006-5a73-4908-8dc5-69641524359e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:42.725135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a7006-5a73-4908-8dc5-69641524359e None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:42.726494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 520/1042] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:42.731479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd44e-372a-42a2-b5de-5573a64fda68 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:42.731768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd44e-372a-42a2-b5de-5573a64fda68 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:42.732050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd44e-372a-42a2-b5de-5573a64fda68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:42.750937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b65dd44e-372a-42a2-b5de-5573a64fda68 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:42.752128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b65dd44e-372a-42a2-b5de-5573a64fda68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:42.821277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 523/1043] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:43.083667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a7006-5a73-4908-8dc5-69641524359e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:43.191784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a7006-5a73-4908-8dc5-69641524359e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:43.229518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:43.230148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.230423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.230689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.241549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:43.246575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:43.247527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:43.249026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 521/1044] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Oct 4 08:39:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:43.259913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dd44e-372a-42a2-b5de-5573a64fda68 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:43.354092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dd44e-372a-42a2-b5de-5573a64fda68 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:43.379671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:43.389621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:43.390227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.390501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.390768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.391429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:43.391727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:43.393103 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:43.401479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 524/1045] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Oct 4 08:39:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:43.407039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:43.416559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:43.514773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095453e9-2b4e-436e-b5ae-4ba6552139d8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:43.549400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:43.550063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.550336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.550601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.560864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:43.562295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:43.577169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:43.578230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:43.579538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:43.579538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:43.579538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:43.579538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 522/1046] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:43.584361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:43.685876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddf57c-bd33-4ac8-89aa-ed97a5e834a2 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:43.720979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a3742-cd1c-458f-8f36-fafd55fa294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:43.721612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a3742-cd1c-458f-8f36-fafd55fa294b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.721888 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a3742-cd1c-458f-8f36-fafd55fa294b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.722156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a3742-cd1c-458f-8f36-fafd55fa294b None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.722886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a3742-cd1c-458f-8f36-fafd55fa294b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:43.723177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a3742-cd1c-458f-8f36-fafd55fa294b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:43.724613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a3742-cd1c-458f-8f36-fafd55fa294b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:43.738503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 525/1047] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:43.744335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:43.810796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:43.848444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb25a-5f79-4ca9-8a8f-c5511ea02235 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.848767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb25a-5f79-4ca9-8a8f-c5511ea02235 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.848980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb25a-5f79-4ca9-8a8f-c5511ea02235 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.850461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 526/1048] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:43.855104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e45f8a-c133-41ec-a423-98e7ce7e4a8d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.855385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e45f8a-c133-41ec-a423-98e7ce7e4a8d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.855653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e45f8a-c133-41ec-a423-98e7ce7e4a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.856700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a6632-66d5-4c26-8f09-71c30ad4097f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:43.874286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-62e45f8a-c133-41ec-a423-98e7ce7e4a8d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:43.875469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-62e45f8a-c133-41ec-a423-98e7ce7e4a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:43.892915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:43.893806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.894231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.894659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.902378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:43.903817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:43.915840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:43.916969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:43.918263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:43.920063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 523/1049] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:39:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:43.923642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:43.926634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:43.955991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 527/1050] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:44.042059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31ddf6-d1fc-45c3-9155-8df77197cd45 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.060347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.078740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca8f91-af51-4cc0-a683-174710289488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.079356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8f91-af51-4cc0-a683-174710289488 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.079634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8f91-af51-4cc0-a683-174710289488 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.079911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8f91-af51-4cc0-a683-174710289488 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.080719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8f91-af51-4cc0-a683-174710289488 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:44.081038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8f91-af51-4cc0-a683-174710289488 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.082687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8f91-af51-4cc0-a683-174710289488 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.101018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 524/1051] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:39:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:44.107046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e45f8a-c133-41ec-a423-98e7ce7e4a8d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.113578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3634f0d-b339-478a-b5f0-6a7759c01bd0 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.113869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3634f0d-b339-478a-b5f0-6a7759c01bd0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.114147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3634f0d-b339-478a-b5f0-6a7759c01bd0 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.115448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 525/1052] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:44.120825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada0ab1-3d8d-41e5-912c-231ef44281a5 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.121104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada0ab1-3d8d-41e5-912c-231ef44281a5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.121670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada0ab1-3d8d-41e5-912c-231ef44281a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.140077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-bada0ab1-3d8d-41e5-912c-231ef44281a5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:44.141262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-bada0ab1-3d8d-41e5-912c-231ef44281a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:44.209099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 526/1053] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:44.218663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e45f8a-c133-41ec-a423-98e7ce7e4a8d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.221919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada0ab1-3d8d-41e5-912c-231ef44281a5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.253923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.254583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.254859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.255124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.264864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.266321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:44.281923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:44.283244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:44.284630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:44.284630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:44.284630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:44.284630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 528/1054] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:44.288757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.321302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.327001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada0ab1-3d8d-41e5-912c-231ef44281a5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.335219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.364571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.365192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.365464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.365752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] PATH_INFO: `/v3/services/image` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.366325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.367784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:44.376658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:44.377666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:39:44.378542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:44.382860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 527/1055] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:44.383685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.397325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5558718-5d35-49e5-ada6-a99637b3e257 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.432890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.433496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.433777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.434047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.434687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:44.434986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.436412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.448465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 529/1056] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:44.453733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.489469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7d946-098a-4962-a698-f8f1e338d86e None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.522147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.522747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.523034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.523303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.523976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] RBAC: Adding query filter params (name=image) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:44.524276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.525657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.530298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 528/1057] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:44.531055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.549194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.560126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a306b97-e54d-4bb8-a93a-1fbe2f6f08d5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.566025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.594661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.595284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.595561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.595830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.602593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.604126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:44.616147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:44.617249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:44.618247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 530/1058] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:44.618776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.635081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af341c2-874a-45d8-aede-0980124f0ff9 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.667978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.668630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.668882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.669177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.670291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.671725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.673330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 529/1059] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:44.674075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.721697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d5294-d2df-40f5-95bc-699fcb3166b9 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.756133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.756662 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.756935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.757201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.757824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] RBAC: Adding query filter params (type=image) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:44.758117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.759448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.764377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 531/1060] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:44.765380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.772993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc15017-c3fc-46df-9e25-04fdfc827e25 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.774533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.780870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.807157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.807980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.808217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.808481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.809214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:44.809522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.811034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.823088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 530/1061] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:44.823928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.878478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46d4ca-963c-4cec-8072-05085b369265 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.912258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.912836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.913113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.913382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.914035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.915465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:44.934066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34932cf8-b859-4fba-9946-a00a6cad94a1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.936789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 532/1062] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:39:44 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:44.948135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.977676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d578b37-a001-4904-ad72-dae336acd3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:44.978341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d578b37-a001-4904-ad72-dae336acd3c4 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.978618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d578b37-a001-4904-ad72-dae336acd3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.978893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d578b37-a001-4904-ad72-dae336acd3c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.979585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d578b37-a001-4904-ad72-dae336acd3c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:44.981031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d578b37-a001-4904-ad72-dae336acd3c4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:45.020028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 531/1063] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:39:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:39:45.041157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:45.082122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1be58-be0c-4553-9c09-944564532a6a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:45.130261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5632c-722f-44ba-bb84-1321dcbc0059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:45.130942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5632c-722f-44ba-bb84-1321dcbc0059 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:45.131216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5632c-722f-44ba-bb84-1321dcbc0059 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:45.133378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5632c-722f-44ba-bb84-1321dcbc0059 None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/bf149f246c1644aa85c4c162a2e07984/roles/96701250d41d426ca366754cd19d50b4` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:45.133809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5632c-722f-44ba-bb84-1321dcbc0059 None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=bf149f246c1644aa85c4c162a2e07984, role_id=96701250d41d426ca366754cd19d50b4)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:45.135260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5632c-722f-44ba-bb84-1321dcbc0059 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:45.153663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 533/1064] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:39:44 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/bf149f246c1644aa85c4c162a2e07984/roles/96701250d41d426ca366754cd19d50b4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:39:47.163668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbca8a6-9608-4143-b1e9-61148592f497 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.164311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbca8a6-9608-4143-b1e9-61148592f497 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.164311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbca8a6-9608-4143-b1e9-61148592f497 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.167936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 532/1065] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:47.171564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5ea3-8ce4-4041-a96a-e2f2a8e3821a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.172039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5ea3-8ce4-4041-a96a-e2f2a8e3821a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.172246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5ea3-8ce4-4041-a96a-e2f2a8e3821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.184350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a110da-0898-48ec-9ab2-e2ee1b50d7ac None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.184660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a110da-0898-48ec-9ab2-e2ee1b50d7ac None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.184893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a110da-0898-48ec-9ab2-e2ee1b50d7ac None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.186639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 533/1066] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:47.193953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-193a5ea3-8ce4-4041-a96a-e2f2a8e3821a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:47.194106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037b98-a92f-4ab1-8e1d-633216af65d2 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.194106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037b98-a92f-4ab1-8e1d-633216af65d2 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.194106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037b98-a92f-4ab1-8e1d-633216af65d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.194328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-193a5ea3-8ce4-4041-a96a-e2f2a8e3821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:47.210308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-fe037b98-a92f-4ab1-8e1d-633216af65d2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:47.211466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fe037b98-a92f-4ab1-8e1d-633216af65d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:47.226020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.270098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 534/1067] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:47.273891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381592b-cef3-49cb-ab97-5c758c3c5558 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.274177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381592b-cef3-49cb-ab97-5c758c3c5558 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.274416 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381592b-cef3-49cb-ab97-5c758c3c5558 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.275981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 535/1068] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:47.280599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8b76-e3c5-4338-ad80-6258d8711b88 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.280879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8b76-e3c5-4338-ad80-6258d8711b88 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.281135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8b76-e3c5-4338-ad80-6258d8711b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.288352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 534/1069] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:47.299613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0cae8b76-e3c5-4338-ad80-6258d8711b88 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:47.301151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0cae8b76-e3c5-4338-ad80-6258d8711b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:47.351210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.369897 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 536/1070] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:47.536392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe037b98-a92f-4ab1-8e1d-633216af65d2 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:47.552252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8b76-e3c5-4338-ad80-6258d8711b88 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:47.648205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe037b98-a92f-4ab1-8e1d-633216af65d2 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:47.658568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8b76-e3c5-4338-ad80-6258d8711b88 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:47.680479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:47.681107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.681519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.681789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.691502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:47.691685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21017189-8364-4712-a842-a99faab244b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:47.692280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21017189-8364-4712-a842-a99faab244b6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.692557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21017189-8364-4712-a842-a99faab244b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.692827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21017189-8364-4712-a842-a99faab244b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:47.696308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:47.697525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 535/1071] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:47.698607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:47.708972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.749496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.764404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21017189-8364-4712-a842-a99faab244b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:47.765853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21017189-8364-4712-a842-a99faab244b6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:47.797327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c2d3e-4c68-4483-b6b8-0fd568521539 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:47.832115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:47.832786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.833058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.833324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.843271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:47.848029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:47.849210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 536/1072] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Oct 4 08:39:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:47.850100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:47.870855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 537/1073] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:47.872050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21017189-8364-4712-a842-a99faab244b6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:47.919264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.953323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c83bd-833d-4eef-8541-f6143b4cab34 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:47.954940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.970733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21017189-8364-4712-a842-a99faab244b6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:47.990174 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:47.990797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.991052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.991318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.992016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:47.992315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:47.993671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.002085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 537/1074] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:48.002844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.004833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.005460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.005738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.006025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.006674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:48.006982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.008365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.020058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 538/1075] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Oct 4 08:39:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:48.020058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.099856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7438867-9adf-4246-b749-2b22e82667d6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.115238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836bf20-bc71-4eb0-b86e-82ea6212674b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.127692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.134554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.135255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.135522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.135790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.147136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.148588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.150226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.150834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.151113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.151380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] PATH_INFO: `/v3/users/a71e82bcea0046579adb270937a25e38` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:48.162636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:48.164310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:48.165509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:48.165509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:48.165509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:48.165509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 538/1076] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:48.165509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.167605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] RBAC: Authorizing `identity:get_user(user_id=a71e82bcea0046579adb270937a25e38)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.169320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.174273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 539/1077] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/users/a71e82bcea0046579adb270937a25e38 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:48.174273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.180618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.264292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac87cf-929f-4446-8e13-3691b3759d74 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.273549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072f596-9ea5-4e75-a82b-7eccdf07063f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.298040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.298672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.298949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.299221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.305691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.306301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.306580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.306847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.307492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:48.307790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.309187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.309735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.311191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.321436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 540/1078] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:48.322215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:48.325430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:48.326548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:48.327503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:48.327503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:48.327503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:48.327503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 539/1079] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:48.332598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.370677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.408871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.444981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26421121-aaf2-4994-af3f-d130aa210eee None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.448705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a019-6458-4e8c-89fb-af63842ae951 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.482143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.482755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.483057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.483292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.483999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:48.484297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.485479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.485707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.486110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.486380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.486645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.498706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 540/1080] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:48.504538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.573984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.575423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.607168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.614428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04206d-2f60-4362-a428-cc87118c2572 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.650117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.650596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.650893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.651159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.654223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.658235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.659680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:39:48.671114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:48.672164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:48.673146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 541/1081] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:48.678059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.695916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 541/1082] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:48.698740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.775643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c7441-669b-4f0f-85d2-09bfe3abd397 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.794794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3009c5-fc17-40e7-a24e-a010289cf789 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.809489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.810102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.810381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.810651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.811381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:48.811683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.813155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.828550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.829179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.829441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.829745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.830576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 542/1083] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:48.830867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] RBAC: Authorizing `identity:get_project(project_id=9196041e96f44bb89a0928b643f3d971)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.832461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:48.834070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 542/1084] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/projects/9196041e96f44bb89a0928b643f3d971 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:48.836632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.840156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:48.851770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.875097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:48.935842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e143171-8719-40e1-9da8-978b5e880838 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.941217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcedd9f-aac1-43c0-a520-0c2b2f6cdbf5 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:48.969602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.970210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.970488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.971191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.979161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:48.979765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:48.980077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:48.980346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:48.981093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=9196041e96f44bb89a0928b643f3d971, user.id=a71e82bcea0046579adb270937a25e38) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:48.981398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:48.982861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:49.000403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 543/1085] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/role_assignments?user.id=a71e82bcea0046579adb270937a25e38&scope.project.id=9196041e96f44bb89a0928b643f3d971&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:39:49.043925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:49.045433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:49.060999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:49.155853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 543/1086] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 08:39:49.162970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:49.273652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:49.278181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7156b-d778-4d53-8f94-d807bab72b29 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:49.314697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:49.315427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:49.315780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:49.316138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:49.328112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:49.329619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:39:49.344244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:49.345333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:39:49.346342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:39:49.346342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:39:49.346342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:49.350418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 544/1087] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:49.351390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:49.450677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27938daf-273e-4449-800e-f71bda4f0b23 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:49.464143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:49.488149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:49.488963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:49.489237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:49.489502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:49.490144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:49.490445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:49.491802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:49.504865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 544/1088] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:49.510641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:49.619390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e795d96-ca40-42f0-b59e-21316e1daf7e None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:49.654275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:49.654900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:49.655146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:49.655439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:49.662251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:49.663733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:39:49.674803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:49.676182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:49.677624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 545/1089] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:39:49.682015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:49.781665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852a08f-ccc3-489b-91f9-c50bc3159736 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:49.816697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb84fd-8a0a-45d2-bc1b-2046e4d95756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:49.817335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb84fd-8a0a-45d2-bc1b-2046e4d95756 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:49.817608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb84fd-8a0a-45d2-bc1b-2046e4d95756 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:49.817879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb84fd-8a0a-45d2-bc1b-2046e4d95756 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:49.818561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb84fd-8a0a-45d2-bc1b-2046e4d95756 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:49.818861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb84fd-8a0a-45d2-bc1b-2046e4d95756 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:49.820316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb84fd-8a0a-45d2-bc1b-2046e4d95756 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:49.832223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 545/1090] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:49.839184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:49.852850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:49.948378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf33f3-66f0-4122-863e-17bb8544f002 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:49.982476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c57d0f-d340-4377-ad31-473187fbf5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:49.983078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c57d0f-d340-4377-ad31-473187fbf5b0 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:49.983345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c57d0f-d340-4377-ad31-473187fbf5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:49.983608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c57d0f-d340-4377-ad31-473187fbf5b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:49.984361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c57d0f-d340-4377-ad31-473187fbf5b0 None admin] RBAC: Adding query filter params (role.id=96701250d41d426ca366754cd19d50b4, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=bf149f246c1644aa85c4c162a2e07984) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:49.984661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c57d0f-d340-4377-ad31-473187fbf5b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:49.986078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c57d0f-d340-4377-ad31-473187fbf5b0 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:49.992274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 546/1091] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:49 2022] GET /identity/v3/role_assignments?user.id=bf149f246c1644aa85c4c162a2e07984&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=96701250d41d426ca366754cd19d50b4 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:51.115431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f39-55e9-4563-ab83-f2885b3bbcd7 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:51.116011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f39-55e9-4563-ab83-f2885b3bbcd7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:51.116126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce64f39-55e9-4563-ab83-f2885b3bbcd7 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:51.117528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 546/1092] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:51.124598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc9d7-c772-4bd3-b217-31a86de5da1b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:51.125327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc9d7-c772-4bd3-b217-31a86de5da1b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:51.125552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc9d7-c772-4bd3-b217-31a86de5da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:51.151017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-090cc9d7-c772-4bd3-b217-31a86de5da1b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:51.152228 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-090cc9d7-c772-4bd3-b217-31a86de5da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:51.223713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 547/1093] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:51.490188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce64f39-55e9-4563-ab83-f2885b3bbcd7 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:51.528383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:51.608821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce64f39-55e9-4563-ab83-f2885b3bbcd7 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:51.647039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:51.647657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:51.647928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:51.648233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:51.661818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:51.666689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:51.667742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:39:51.669027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 547/1094] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:51.676401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cc9d7-c772-4bd3-b217-31a86de5da1b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:51.679755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:51.784639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cc9d7-c772-4bd3-b217-31a86de5da1b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:51.821056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:51.821671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:51.821945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:51.822229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:51.822912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:51.823210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:51.824621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:51.832833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 548/1095] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:51.839609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:51.954909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:51.965557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439156e-ab23-442c-8765-f17fb556fc9c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:52.001853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:52.002479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:52.002754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:52.003024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] PATH_INFO: `/v3/users/a71e82bcea0046579adb270937a25e38` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:52.004261 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] RBAC: Authorizing `identity:get_user(user_id=a71e82bcea0046579adb270937a25e38)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:52.005695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:52.007301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 548/1096] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:39:51 2022] GET /identity/v3/users/a71e82bcea0046579adb270937a25e38 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:52.016642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:52.091686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:52.133104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2654f-a997-47ae-95cb-1bf5dfd077c1 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:52.166927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0a831-7842-442c-8a47-26587cbe1d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:52.167617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a831-7842-442c-8a47-26587cbe1d41 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:52.167926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a831-7842-442c-8a47-26587cbe1d41 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:52.168189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a831-7842-442c-8a47-26587cbe1d41 None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:52.169413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0a831-7842-442c-8a47-26587cbe1d41 None admin] RBAC: Authorizing `identity:get_project(project_id=9196041e96f44bb89a0928b643f3d971)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:52.170823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0a831-7842-442c-8a47-26587cbe1d41 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:52.172453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 549/1097] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:39:52 2022] GET /identity/v3/projects/9196041e96f44bb89a0928b643f3d971 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:52.178754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:52.312464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f96419-e91e-4775-8d6b-9f56b789f26f None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:52.357165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd03f-9e03-49fc-a8e7-cd26ad51c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:52.357742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd03f-9e03-49fc-a8e7-cd26ad51c33d None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:52.358021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd03f-9e03-49fc-a8e7-cd26ad51c33d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:52.358290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd03f-9e03-49fc-a8e7-cd26ad51c33d None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971/users/a71e82bcea0046579adb270937a25e38/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:52.360339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cd03f-9e03-49fc-a8e7-cd26ad51c33d None admin] RBAC: Authorizing `identity:create_grant(project_id=9196041e96f44bb89a0928b643f3d971, user_id=a71e82bcea0046579adb270937a25e38, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:52.361773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cd03f-9e03-49fc-a8e7-cd26ad51c33d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:52.378295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 549/1098] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:39:52 2022] PUT /identity/v3/projects/9196041e96f44bb89a0928b643f3d971/users/a71e82bcea0046579adb270937a25e38/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:39:54.498378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7bced-f52c-4507-a8e0-197a07319a04 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:54.498960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7bced-f52c-4507-a8e0-197a07319a04 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:54.498960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7bced-f52c-4507-a8e0-197a07319a04 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:54.501003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 550/1099] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:54.506513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398cd16-d372-4521-9ec8-bd24eefe6321 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:54.506959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398cd16-d372-4521-9ec8-bd24eefe6321 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:54.507068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398cd16-d372-4521-9ec8-bd24eefe6321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:54.526413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b398cd16-d372-4521-9ec8-bd24eefe6321 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:54.527583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b398cd16-d372-4521-9ec8-bd24eefe6321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:54.543116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:54.606860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 550/1100] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:54.872310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7bced-f52c-4507-a8e0-197a07319a04 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:55.004920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7bced-f52c-4507-a8e0-197a07319a04 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:55.040292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:55.040923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:55.041198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:55.041460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:55.043529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:55.053098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:39:55.057875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:39:55.058941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:39:55.059830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 551/1101] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:39:55.065368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398cd16-d372-4521-9ec8-bd24eefe6321 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:55.180757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398cd16-d372-4521-9ec8-bd24eefe6321 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:55.218027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:55.218632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:55.218910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:55.219185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:55.219854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:55.220187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:55.221565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:55.230093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 551/1102] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:55.236493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:55.345651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dd202-9462-465b-9bb6-75747d627103 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:55.380717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:55.381358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:55.381652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:55.381917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] PATH_INFO: `/v3/users/a71e82bcea0046579adb270937a25e38` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:55.383128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] RBAC: Authorizing `identity:get_user(user_id=a71e82bcea0046579adb270937a25e38)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:55.384585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:55.386145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 552/1103] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:39:55 2022] GET /identity/v3/users/a71e82bcea0046579adb270937a25e38 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:55.392817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:55.414122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:55.505873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b92c3f-97f4-4541-9e8c-35aecdd150eb None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:55.540763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:55.541378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:55.541627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:55.541891 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:55.614807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:55.616304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:55.631503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:55.719582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 552/1104] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:39:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:39:55.727216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:55.791530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:55.838537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9eaf3-751d-4ca9-8781-ae780d300274 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:55.876317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8704c-b8d7-428b-aa46-8543c8f3efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:55.876929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8704c-b8d7-428b-aa46-8543c8f3efd5 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:55.877199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8704c-b8d7-428b-aa46-8543c8f3efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:55.877469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8704c-b8d7-428b-aa46-8543c8f3efd5 None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:55.878611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8704c-b8d7-428b-aa46-8543c8f3efd5 None admin] RBAC: Authorizing `identity:get_project(project_id=9196041e96f44bb89a0928b643f3d971)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:55.880088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8704c-b8d7-428b-aa46-8543c8f3efd5 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:55.881697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 553/1105] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:39:55 2022] GET /identity/v3/projects/9196041e96f44bb89a0928b643f3d971 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:55.888039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:55.991415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e7b65-922d-464a-877e-78625fae5c56 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:56.004653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:56.036645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a7924-eb27-41df-8686-45dfccfa96c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:56.037237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a7924-eb27-41df-8686-45dfccfa96c3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:56.037576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a7924-eb27-41df-8686-45dfccfa96c3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:56.037846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a7924-eb27-41df-8686-45dfccfa96c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:56.038603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a7924-eb27-41df-8686-45dfccfa96c3 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=9196041e96f44bb89a0928b643f3d971, user.id=a71e82bcea0046579adb270937a25e38) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:39:56.038905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a7924-eb27-41df-8686-45dfccfa96c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:56.040367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a7924-eb27-41df-8686-45dfccfa96c3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:56.046254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 553/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:39:55 2022] GET /identity/v3/role_assignments?user.id=a71e82bcea0046579adb270937a25e38&scope.project.id=9196041e96f44bb89a0928b643f3d971&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:58.189112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e071b-7985-465c-88ec-a3291328d37f None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:58.189695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e071b-7985-465c-88ec-a3291328d37f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:58.189695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e071b-7985-465c-88ec-a3291328d37f None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:58.191034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 554/1107] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:58.197481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdaf79-9702-48a5-8f8f-56c9f38f4e79 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:58.197895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdaf79-9702-48a5-8f8f-56c9f38f4e79 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:58.198073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdaf79-9702-48a5-8f8f-56c9f38f4e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:58.226859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d5bdaf79-9702-48a5-8f8f-56c9f38f4e79 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:58.228134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d5bdaf79-9702-48a5-8f8f-56c9f38f4e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:58.297489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 554/1108] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:39:58.566683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e071b-7985-465c-88ec-a3291328d37f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:58.677513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e071b-7985-465c-88ec-a3291328d37f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:58.706496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:58.713267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead14591-c9a8-4c51-844a-844990c53a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:58.713889 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14591-c9a8-4c51-844a-844990c53a59 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:58.714162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14591-c9a8-4c51-844a-844990c53a59 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:58.714432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14591-c9a8-4c51-844a-844990c53a59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:58.715548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14591-c9a8-4c51-844a-844990c53a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:58.716990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14591-c9a8-4c51-844a-844990c53a59 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:58.718580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 555/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:39:58.725242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdaf79-9702-48a5-8f8f-56c9f38f4e79 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:58.837890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdaf79-9702-48a5-8f8f-56c9f38f4e79 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:58.849030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:58.885193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69de174-e5c1-455d-aaef-b86ae8ed1f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:58.885816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69de174-e5c1-455d-aaef-b86ae8ed1f1f None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:58.886093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69de174-e5c1-455d-aaef-b86ae8ed1f1f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:58.886357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69de174-e5c1-455d-aaef-b86ae8ed1f1f None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:58.886999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69de174-e5c1-455d-aaef-b86ae8ed1f1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:58.888479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69de174-e5c1-455d-aaef-b86ae8ed1f1f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:58.914726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 555/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:39:58 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:40:01.076800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae0b5f-3ccf-439b-96f3-a7bc694e9d85 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:01.077379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae0b5f-3ccf-439b-96f3-a7bc694e9d85 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:01.077379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae0b5f-3ccf-439b-96f3-a7bc694e9d85 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:01.078684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 556/1111] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:01.085035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed295-2e6a-421a-80d0-7df1a8b468a9 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:01.085764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed295-2e6a-421a-80d0-7df1a8b468a9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:01.085937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed295-2e6a-421a-80d0-7df1a8b468a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:01.110700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-245ed295-2e6a-421a-80d0-7df1a8b468a9 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:01.112073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-245ed295-2e6a-421a-80d0-7df1a8b468a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:01.190959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 556/1112] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:01.467666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae0b5f-3ccf-439b-96f3-a7bc694e9d85 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:01.595583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae0b5f-3ccf-439b-96f3-a7bc694e9d85 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:01.632864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:01.633488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:01.633770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:01.634042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] PATH_INFO: `/v3/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:01.635493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] RBAC: Authorizing `identity:get_role(role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:01.636924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:01.638252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 557/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:01 2022] GET /identity/v3/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:01.647128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed295-2e6a-421a-80d0-7df1a8b468a9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:01.741511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:01.755576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed295-2e6a-421a-80d0-7df1a8b468a9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:01.792187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:01.792810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:01.793085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:01.793354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] PATH_INFO: `/v3/users/389a2e1aed564b09b8239e7d7d46aabb` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:01.813455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] RBAC: Authorizing `identity:get_user(user_id=389a2e1aed564b09b8239e7d7d46aabb)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:01.815637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:01.817942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 557/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:01 2022] GET /identity/v3/users/389a2e1aed564b09b8239e7d7d46aabb => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:40:01.824000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:01.892778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:01.927385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fabfaa-b729-43a5-a825-34b633ecf7fc None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:01.962049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:01.962642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:01.962912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:01.963180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:02.031364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:02.032842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:02.101804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:02.138399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 558/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:40:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:40:02.144707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:02.248270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a10209-cd8d-4b87-9a2e-2c1eeff31c06 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:02.284382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d3c1-e696-4b83-b2ea-0a91385e2617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:02.284942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d3c1-e696-4b83-b2ea-0a91385e2617 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:02.285212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d3c1-e696-4b83-b2ea-0a91385e2617 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:02.285478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d3c1-e696-4b83-b2ea-0a91385e2617 None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:02.286678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598d3c1-e696-4b83-b2ea-0a91385e2617 None admin] RBAC: Authorizing `identity:get_project(project_id=9196041e96f44bb89a0928b643f3d971)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:02.288124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598d3c1-e696-4b83-b2ea-0a91385e2617 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:02.289712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 558/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:02 2022] GET /identity/v3/projects/9196041e96f44bb89a0928b643f3d971 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:02.296123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:02.406766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a1aa1-d8e2-4d33-a585-fe63abd372d1 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:02.441741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f13ff-8d10-4eb8-8769-f92c7c6dadf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:02.442422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f13ff-8d10-4eb8-8769-f92c7c6dadf3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:02.442658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f13ff-8d10-4eb8-8769-f92c7c6dadf3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:02.442928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f13ff-8d10-4eb8-8769-f92c7c6dadf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:02.443668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f13ff-8d10-4eb8-8769-f92c7c6dadf3 None admin] RBAC: Adding query filter params (role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=9196041e96f44bb89a0928b643f3d971, user.id=389a2e1aed564b09b8239e7d7d46aabb) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:02.444005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f13ff-8d10-4eb8-8769-f92c7c6dadf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:02.445408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f13ff-8d10-4eb8-8769-f92c7c6dadf3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:02.450901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 559/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:40:02 2022] GET /identity/v3/role_assignments?user.id=389a2e1aed564b09b8239e7d7d46aabb&scope.project.id=9196041e96f44bb89a0928b643f3d971&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:04.526953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896fa45-32a2-466d-84e7-1cbb47ebf207 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:04.527518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896fa45-32a2-466d-84e7-1cbb47ebf207 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:04.527518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896fa45-32a2-466d-84e7-1cbb47ebf207 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:04.528869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 559/1118] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:04.535060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc1ff5-1469-46e2-9172-565053dfc5d9 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:04.535520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc1ff5-1469-46e2-9172-565053dfc5d9 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:04.535621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc1ff5-1469-46e2-9172-565053dfc5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:04.560914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b4dc1ff5-1469-46e2-9172-565053dfc5d9 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:04.562127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b4dc1ff5-1469-46e2-9172-565053dfc5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:04.585916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:04.645605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 560/1119] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:04.909521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896fa45-32a2-466d-84e7-1cbb47ebf207 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:04.931490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:05.022919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896fa45-32a2-466d-84e7-1cbb47ebf207 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:05.062583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:05.063204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:05.063481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:05.063750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] PATH_INFO: `/v3/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:05.065295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] RBAC: Authorizing `identity:get_role(role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:05.066661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:05.068052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 560/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:04 2022] GET /identity/v3/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:05.077495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc1ff5-1469-46e2-9172-565053dfc5d9 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:05.195994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc1ff5-1469-46e2-9172-565053dfc5d9 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:05.230745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:05.231359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:05.231635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:05.231931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] PATH_INFO: `/v3/users/389a2e1aed564b09b8239e7d7d46aabb` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:05.233083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] RBAC: Authorizing `identity:get_user(user_id=389a2e1aed564b09b8239e7d7d46aabb)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:05.234512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:05.236116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 561/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:05 2022] GET /identity/v3/users/389a2e1aed564b09b8239e7d7d46aabb => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:05.242430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:05.329167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:05.345267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d0cde-57ee-4cb4-a428-0ffbffbe2986 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:05.380251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352606db-866c-457a-b47f-b6e347e10565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:05.380903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352606db-866c-457a-b47f-b6e347e10565 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:05.381208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352606db-866c-457a-b47f-b6e347e10565 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:05.381424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352606db-866c-457a-b47f-b6e347e10565 None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:05.382681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352606db-866c-457a-b47f-b6e347e10565 None admin] RBAC: Authorizing `identity:get_project(project_id=9196041e96f44bb89a0928b643f3d971)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:05.384139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352606db-866c-457a-b47f-b6e347e10565 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:05.385737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 561/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:05 2022] GET /identity/v3/projects/9196041e96f44bb89a0928b643f3d971 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:05.395329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:05.424846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:05.508202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06a6dd-bea4-4078-b2ce-0a69bef5f4ba None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:05.542752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039f07-d266-4a7c-a897-00d624290bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:05.543356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039f07-d266-4a7c-a897-00d624290bdf None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:05.543658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039f07-d266-4a7c-a897-00d624290bdf None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:05.543916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039f07-d266-4a7c-a897-00d624290bdf None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971/users/389a2e1aed564b09b8239e7d7d46aabb/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:05.546465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039f07-d266-4a7c-a897-00d624290bdf None admin] RBAC: Authorizing `identity:create_grant(project_id=9196041e96f44bb89a0928b643f3d971, user_id=389a2e1aed564b09b8239e7d7d46aabb, role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:05.547904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039f07-d266-4a7c-a897-00d624290bdf None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:05.563627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 562/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:40:05 2022] PUT /identity/v3/projects/9196041e96f44bb89a0928b643f3d971/users/389a2e1aed564b09b8239e7d7d46aabb/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:40:07.652697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257524-4e34-405f-9e66-8a73ac1081ef None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:07.653534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257524-4e34-405f-9e66-8a73ac1081ef None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:07.653534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257524-4e34-405f-9e66-8a73ac1081ef None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:07.655665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 562/1124] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:07.664605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b34a5-dc43-4de0-8639-3001e0cc01c4 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:07.665016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b34a5-dc43-4de0-8639-3001e0cc01c4 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:07.665211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b34a5-dc43-4de0-8639-3001e0cc01c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:07.694811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-fc5b34a5-dc43-4de0-8639-3001e0cc01c4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:07.696009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fc5b34a5-dc43-4de0-8639-3001e0cc01c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:07.774802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 563/1125] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:08.050840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d257524-4e34-405f-9e66-8a73ac1081ef None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:08.164614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d257524-4e34-405f-9e66-8a73ac1081ef None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:08.207638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:08.208453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:08.208724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:08.208990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] PATH_INFO: `/v3/roles/2415eb0424704d7bb45c2b5f84cad4ee` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:08.210491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] RBAC: Authorizing `identity:get_role(role_id=2415eb0424704d7bb45c2b5f84cad4ee)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:08.211864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:08.213171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 563/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:08 2022] GET /identity/v3/roles/2415eb0424704d7bb45c2b5f84cad4ee => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:08.219064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b34a5-dc43-4de0-8639-3001e0cc01c4 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:08.238794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:08.336203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b34a5-dc43-4de0-8639-3001e0cc01c4 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:08.370786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:08.371410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:08.372012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:08.372312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] PATH_INFO: `/v3/users/389a2e1aed564b09b8239e7d7d46aabb` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:08.373566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] RBAC: Authorizing `identity:get_user(user_id=389a2e1aed564b09b8239e7d7d46aabb)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:08.374983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:08.376580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 564/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:08 2022] GET /identity/v3/users/389a2e1aed564b09b8239e7d7d46aabb => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:08.383371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:08.414033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:08.495498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441fb7a-3147-489e-befc-3fc8dad8bf99 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:08.533496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:08.534085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:08.534364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:08.534633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:08.602201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:08.603638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:08.629737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:08.708405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 564/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:40:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:40:08.716159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:08.807243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:08.822715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a4ee2-1515-4aef-95e4-b0b08693032c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:08.856394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b610d-4f91-4bb1-8d2e-86ddd7a0e5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:08.857007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b610d-4f91-4bb1-8d2e-86ddd7a0e5b1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:08.857280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b610d-4f91-4bb1-8d2e-86ddd7a0e5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:08.857541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b610d-4f91-4bb1-8d2e-86ddd7a0e5b1 None admin] PATH_INFO: `/v3/projects/9196041e96f44bb89a0928b643f3d971` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:08.858661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b610d-4f91-4bb1-8d2e-86ddd7a0e5b1 None admin] RBAC: Authorizing `identity:get_project(project_id=9196041e96f44bb89a0928b643f3d971)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:08.860117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b610d-4f91-4bb1-8d2e-86ddd7a0e5b1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:08.861717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 565/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:08 2022] GET /identity/v3/projects/9196041e96f44bb89a0928b643f3d971 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:08.872534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:08.983848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bf540-5a78-4650-9a9c-6c20f59847f3 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:09.006644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:09.020047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eed5b-308a-4413-887c-7f38892c7887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:09.020667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eed5b-308a-4413-887c-7f38892c7887 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:09.020944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eed5b-308a-4413-887c-7f38892c7887 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:09.021225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eed5b-308a-4413-887c-7f38892c7887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:09.021932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eed5b-308a-4413-887c-7f38892c7887 None admin] RBAC: Adding query filter params (role.id=2415eb0424704d7bb45c2b5f84cad4ee, scope.project.id=9196041e96f44bb89a0928b643f3d971, user.id=389a2e1aed564b09b8239e7d7d46aabb) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:09.022239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eed5b-308a-4413-887c-7f38892c7887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:09.023669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eed5b-308a-4413-887c-7f38892c7887 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:09.029374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 565/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:40:08 2022] GET /identity/v3/role_assignments?user.id=389a2e1aed564b09b8239e7d7d46aabb&scope.project.id=9196041e96f44bb89a0928b643f3d971&role.id=2415eb0424704d7bb45c2b5f84cad4ee => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:11.183207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20572bb5-ea4a-4162-a269-f83a22da0e60 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:11.184136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20572bb5-ea4a-4162-a269-f83a22da0e60 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:11.184136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20572bb5-ea4a-4162-a269-f83a22da0e60 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:11.185808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 566/1131] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:11.192537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef967e-a007-40ba-9ac3-41b95bcdfd9a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:11.192932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef967e-a007-40ba-9ac3-41b95bcdfd9a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:11.193074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef967e-a007-40ba-9ac3-41b95bcdfd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:11.212853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b1ef967e-a007-40ba-9ac3-41b95bcdfd9a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:11.214048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b1ef967e-a007-40ba-9ac3-41b95bcdfd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:11.291813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 566/1132] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:11.555512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20572bb5-ea4a-4162-a269-f83a22da0e60 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:11.672211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20572bb5-ea4a-4162-a269-f83a22da0e60 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:11.710767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0970c27-7e2f-4ad2-9c52-30b54d7b7cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:11.711389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0970c27-7e2f-4ad2-9c52-30b54d7b7cbd None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:11.711667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0970c27-7e2f-4ad2-9c52-30b54d7b7cbd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:11.711941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0970c27-7e2f-4ad2-9c52-30b54d7b7cbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:11.713202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0970c27-7e2f-4ad2-9c52-30b54d7b7cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:11.714631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0970c27-7e2f-4ad2-9c52-30b54d7b7cbd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:11.716236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 567/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:40:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:11.724749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef967e-a007-40ba-9ac3-41b95bcdfd9a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:11.843325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef967e-a007-40ba-9ac3-41b95bcdfd9a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:11.861203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:11.881330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05e772-e5ad-444c-a96b-eb8be1f1acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:11.881975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05e772-e5ad-444c-a96b-eb8be1f1acee None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:11.882248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05e772-e5ad-444c-a96b-eb8be1f1acee None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:11.882514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05e772-e5ad-444c-a96b-eb8be1f1acee None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:11.883159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05e772-e5ad-444c-a96b-eb8be1f1acee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:11.884626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05e772-e5ad-444c-a96b-eb8be1f1acee None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:11.919325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 567/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:40:11 2022] POST /identity/v3/projects => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:40:14.068289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccebc6-7e06-45d0-8d4f-e89b5c656bd0 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:14.069181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccebc6-7e06-45d0-8d4f-e89b5c656bd0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:14.069181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccebc6-7e06-45d0-8d4f-e89b5c656bd0 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:14.071340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 568/1135] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:14.077703 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c1ff-77fc-423a-9aa6-137775db4f28 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:14.078426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c1ff-77fc-423a-9aa6-137775db4f28 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:14.078720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326c1ff-77fc-423a-9aa6-137775db4f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:14.100165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9326c1ff-77fc-423a-9aa6-137775db4f28 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:14.101359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9326c1ff-77fc-423a-9aa6-137775db4f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:14.173698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 568/1136] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:14.436831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccebc6-7e06-45d0-8d4f-e89b5c656bd0 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:14.467082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:14.557721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccebc6-7e06-45d0-8d4f-e89b5c656bd0 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:14.592684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1bbd5-5d59-4b80-8274-e69b33a8db10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:14.593316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1bbd5-5d59-4b80-8274-e69b33a8db10 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:14.593599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1bbd5-5d59-4b80-8274-e69b33a8db10 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:14.593868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1bbd5-5d59-4b80-8274-e69b33a8db10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:14.594992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1bbd5-5d59-4b80-8274-e69b33a8db10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:14.596446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1bbd5-5d59-4b80-8274-e69b33a8db10 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:14.598020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 569/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:40:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:14.604187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326c1ff-77fc-423a-9aa6-137775db4f28 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:14.722927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:14.727014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326c1ff-77fc-423a-9aa6-137775db4f28 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:14.762897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af9e9d-72ca-4986-a525-3a674f1dd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:14.763499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9e9d-72ca-4986-a525-3a674f1dd8b2 None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:14.763776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9e9d-72ca-4986-a525-3a674f1dd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:14.764074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af9e9d-72ca-4986-a525-3a674f1dd8b2 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:14.764712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af9e9d-72ca-4986-a525-3a674f1dd8b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:14.766132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af9e9d-72ca-4986-a525-3a674f1dd8b2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:14.789096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 569/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:40:14 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:40:16.940359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9fdf0-d712-4606-a41a-c105f997d6a2 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:16.941195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9fdf0-d712-4606-a41a-c105f997d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:16.941195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9fdf0-d712-4606-a41a-c105f997d6a2 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:16.943480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 570/1139] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:16.949055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b6118-7e4f-4431-94b0-183ae7e3695e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:16.949488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b6118-7e4f-4431-94b0-183ae7e3695e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:16.949603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b6118-7e4f-4431-94b0-183ae7e3695e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:16.969087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8c4b6118-7e4f-4431-94b0-183ae7e3695e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:16.970572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8c4b6118-7e4f-4431-94b0-183ae7e3695e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:17.042710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 570/1140] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:17.332699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9fdf0-d712-4606-a41a-c105f997d6a2 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:17.431559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:17.445067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9fdf0-d712-4606-a41a-c105f997d6a2 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:17.496499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:17.497438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:17.497717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:17.498016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:17.508561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:40:17.513320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:40:17.514423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:40:17.515825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 571/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:40:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:40:17.522923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b6118-7e4f-4431-94b0-183ae7e3695e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:17.609926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:17.628985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b6118-7e4f-4431-94b0-183ae7e3695e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:17.670397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:17.670996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:17.671270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:17.671535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:17.672223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:17.672524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:17.673862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:17.682106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 571/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:17.688916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:17.809749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34927a-da31-4fb9-9b51-19eed3fa4ae6 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:17.842001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:17.842613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:17.842884 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:17.843148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] PATH_INFO: `/v3/users/8929b9ec574b42f5b509a3d3bca8aef1` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:17.856736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] RBAC: Authorizing `identity:get_user(user_id=8929b9ec574b42f5b509a3d3bca8aef1)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:17.858178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:17.859763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 572/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:17 2022] GET /identity/v3/users/8929b9ec574b42f5b509a3d3bca8aef1 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:17.865842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:17.988459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43a149-541e-4e67-83b9-d5b05b7de1c4 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:18.026336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:18.026948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:18.027228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:18.027497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:18.043335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:18.098038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:18.099548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:18.202695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 572/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:40:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:40:18.209591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:18.227187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:18.324842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcfec6-d77d-467c-9459-142009e3ec06 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:18.371399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e7204-2ad7-4cdf-b5dc-04e90421f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:18.372391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7204-2ad7-4cdf-b5dc-04e90421f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:18.372667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7204-2ad7-4cdf-b5dc-04e90421f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:18.372930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7204-2ad7-4cdf-b5dc-04e90421f1a8 None admin] PATH_INFO: `/v3/projects/f0f258e286244859b81579d42d1922d3` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:18.374252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e7204-2ad7-4cdf-b5dc-04e90421f1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=f0f258e286244859b81579d42d1922d3)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:18.375673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e7204-2ad7-4cdf-b5dc-04e90421f1a8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:18.377332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 573/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:18 2022] GET /identity/v3/projects/f0f258e286244859b81579d42d1922d3 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:40:18.383821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:18.433129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:18.490789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715e63-010f-45d4-bd45-b7bef6ef1d4c None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:18.530093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84727fdf-e495-481e-ae1f-7cc4e22fe92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:18.530668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727fdf-e495-481e-ae1f-7cc4e22fe92f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:18.530942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727fdf-e495-481e-ae1f-7cc4e22fe92f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:18.531213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84727fdf-e495-481e-ae1f-7cc4e22fe92f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:18.532079 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84727fdf-e495-481e-ae1f-7cc4e22fe92f None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=f0f258e286244859b81579d42d1922d3, user.id=8929b9ec574b42f5b509a3d3bca8aef1) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:18.532385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84727fdf-e495-481e-ae1f-7cc4e22fe92f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:18.533832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84727fdf-e495-481e-ae1f-7cc4e22fe92f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:18.539918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 573/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:40:18 2022] GET /identity/v3/role_assignments?user.id=8929b9ec574b42f5b509a3d3bca8aef1&scope.project.id=f0f258e286244859b81579d42d1922d3&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:20.646714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27186b66-f428-42d0-9825-8859d5423ff7 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:20.647283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27186b66-f428-42d0-9825-8859d5423ff7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:20.647394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27186b66-f428-42d0-9825-8859d5423ff7 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:20.649507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 574/1147] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:20.658091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fab23-4f17-41a1-a2cb-6b387819b8ce None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:20.658494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fab23-4f17-41a1-a2cb-6b387819b8ce None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:20.658670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fab23-4f17-41a1-a2cb-6b387819b8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:20.690385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-604fab23-4f17-41a1-a2cb-6b387819b8ce None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:20.691575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-604fab23-4f17-41a1-a2cb-6b387819b8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:20.767624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 574/1148] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:21.034826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27186b66-f428-42d0-9825-8859d5423ff7 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:21.120740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:21.148404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27186b66-f428-42d0-9825-8859d5423ff7 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:21.187708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:21.188363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:21.188638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:21.188900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:21.199413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:40:21.204176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:40:21.205431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:40:21.206654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 575/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:40:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:40:21.212111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fab23-4f17-41a1-a2cb-6b387819b8ce None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:21.266544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:21.333871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fab23-4f17-41a1-a2cb-6b387819b8ce None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:21.375168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:21.375765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:21.376072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:21.376324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:21.376988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:21.377279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:21.378646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:21.386992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 575/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:40:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:40:21.393635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:21.512545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a338e9c-c330-40b0-8969-accfa891c840 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:21.547536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:21.548185 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:21.548457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:21.548722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] PATH_INFO: `/v3/users/8929b9ec574b42f5b509a3d3bca8aef1` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:21.549876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] RBAC: Authorizing `identity:get_user(user_id=8929b9ec574b42f5b509a3d3bca8aef1)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:21.551291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:21.552925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 576/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:21 2022] GET /identity/v3/users/8929b9ec574b42f5b509a3d3bca8aef1 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:21.558958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:21.666479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10881f-6fa3-492f-8a50-528917190497 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:21.668204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:21.702895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1c622-7a17-4767-b248-878195e6ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:21.703508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1c622-7a17-4767-b248-878195e6ce4d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:21.703780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1c622-7a17-4767-b248-878195e6ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:21.704075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1c622-7a17-4767-b248-878195e6ce4d None admin] PATH_INFO: `/v3/projects/f0f258e286244859b81579d42d1922d3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:21.705221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1c622-7a17-4767-b248-878195e6ce4d None admin] RBAC: Authorizing `identity:get_project(project_id=f0f258e286244859b81579d42d1922d3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:21.706630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1c622-7a17-4767-b248-878195e6ce4d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:21.708294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 576/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:21 2022] GET /identity/v3/projects/f0f258e286244859b81579d42d1922d3 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:21.714423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:21.718593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:21.820099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee44a-64a7-43fd-8acd-0e06f364ae31 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:21.855358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af473eef-4024-4b94-ab2a-c2cec91c9894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:21.856030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af473eef-4024-4b94-ab2a-c2cec91c9894 None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:21.856308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af473eef-4024-4b94-ab2a-c2cec91c9894 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:21.856580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af473eef-4024-4b94-ab2a-c2cec91c9894 None admin] PATH_INFO: `/v3/projects/f0f258e286244859b81579d42d1922d3/users/8929b9ec574b42f5b509a3d3bca8aef1/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:21.858712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af473eef-4024-4b94-ab2a-c2cec91c9894 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0f258e286244859b81579d42d1922d3, user_id=8929b9ec574b42f5b509a3d3bca8aef1, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:21.860170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af473eef-4024-4b94-ab2a-c2cec91c9894 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:21.878459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 577/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:40:21 2022] PUT /identity/v3/projects/f0f258e286244859b81579d42d1922d3/users/8929b9ec574b42f5b509a3d3bca8aef1/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:40:23.977426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca6f98-7ae3-4a0d-9673-4876070cf739 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:23.977904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca6f98-7ae3-4a0d-9673-4876070cf739 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:23.978005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca6f98-7ae3-4a0d-9673-4876070cf739 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:23.980026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 577/1154] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:23.985298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094f9a9-6206-40f0-ad12-96fed10b6329 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:23.985733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094f9a9-6206-40f0-ad12-96fed10b6329 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:23.985827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094f9a9-6206-40f0-ad12-96fed10b6329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:24.006033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-8094f9a9-6206-40f0-ad12-96fed10b6329 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:24.007233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8094f9a9-6206-40f0-ad12-96fed10b6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:24.069758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:24.083152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 578/1155] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:24.375815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca6f98-7ae3-4a0d-9673-4876070cf739 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:24.495019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca6f98-7ae3-4a0d-9673-4876070cf739 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:24.536636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:24.537607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:24.538071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:24.538523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:24.561643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:40:24.566517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:40:24.567681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:40:24.568909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 578/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:40:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:40:24.573911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094f9a9-6206-40f0-ad12-96fed10b6329 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:24.677874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094f9a9-6206-40f0-ad12-96fed10b6329 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:24.713362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:24.713959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:24.714230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:24.714497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:24.715164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:24.715473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:24.716844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:24.724954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 579/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:40:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:24.734502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:24.775347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:24.853619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030e719-8fa2-4f27-8336-3324b0d68502 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:24.900153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:24.900771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:24.901047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:24.901311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] PATH_INFO: `/v3/users/8929b9ec574b42f5b509a3d3bca8aef1` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:24.902549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] RBAC: Authorizing `identity:get_user(user_id=8929b9ec574b42f5b509a3d3bca8aef1)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:24.903990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:24.905569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 579/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:24 2022] GET /identity/v3/users/8929b9ec574b42f5b509a3d3bca8aef1 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:24.913890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:24.976865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:25.027837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5959e3-fcef-4863-a8e9-a42d38d9dfd8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:25.062092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:25.062688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:25.062960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:25.063223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:25.145273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:25.146743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:25.211302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:25.259972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 580/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:40:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:40:25.266116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:25.367366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:25.380705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b735289-e271-4c38-92de-b5c605b50f74 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:25.434762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9d751-bfe4-47bd-b2ff-c8be8a5f464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:25.435382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d751-bfe4-47bd-b2ff-c8be8a5f464f None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:25.435658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d751-bfe4-47bd-b2ff-c8be8a5f464f None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:25.435927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d751-bfe4-47bd-b2ff-c8be8a5f464f None admin] PATH_INFO: `/v3/projects/f0f258e286244859b81579d42d1922d3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:25.437196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9d751-bfe4-47bd-b2ff-c8be8a5f464f None admin] RBAC: Authorizing `identity:get_project(project_id=f0f258e286244859b81579d42d1922d3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:25.438627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9d751-bfe4-47bd-b2ff-c8be8a5f464f None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:25.440299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 580/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:25 2022] GET /identity/v3/projects/f0f258e286244859b81579d42d1922d3 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:25.447562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:25.558110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871e7cf-f69d-46ba-a5bd-45bcd5e43886 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:25.593339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7e84f-7e0b-4c58-9a13-f8da2163fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:25.593892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e84f-7e0b-4c58-9a13-f8da2163fecc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:25.594204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e84f-7e0b-4c58-9a13-f8da2163fecc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:25.594471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e84f-7e0b-4c58-9a13-f8da2163fecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:25.595191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7e84f-7e0b-4c58-9a13-f8da2163fecc None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=f0f258e286244859b81579d42d1922d3, user.id=8929b9ec574b42f5b509a3d3bca8aef1) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:25.595501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7e84f-7e0b-4c58-9a13-f8da2163fecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:25.596949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7e84f-7e0b-4c58-9a13-f8da2163fecc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:25.603317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 581/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:40:25 2022] GET /identity/v3/role_assignments?user.id=8929b9ec574b42f5b509a3d3bca8aef1&scope.project.id=f0f258e286244859b81579d42d1922d3&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:27.741573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659b4e-6bac-49ae-a940-a1326e2df3eb None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:27.742461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659b4e-6bac-49ae-a940-a1326e2df3eb None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:27.742461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659b4e-6bac-49ae-a940-a1326e2df3eb None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:27.744657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 581/1162] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:27.751649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a0354-4d22-49cf-93e2-9807e5ba6e54 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:27.752110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a0354-4d22-49cf-93e2-9807e5ba6e54 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:27.752247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a0354-4d22-49cf-93e2-9807e5ba6e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:27.773338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c07a0354-4d22-49cf-93e2-9807e5ba6e54 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:27.774525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c07a0354-4d22-49cf-93e2-9807e5ba6e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:27.790500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:27.846953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 582/1163] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:28.111209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659b4e-6bac-49ae-a940-a1326e2df3eb None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:28.220732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659b4e-6bac-49ae-a940-a1326e2df3eb None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:28.256053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:28.256643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:28.256918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:28.257189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:28.305631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:28.351239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:28.352706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:28.451389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 582/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:40:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:40:28.460655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a0354-4d22-49cf-93e2-9807e5ba6e54 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:28.572558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a0354-4d22-49cf-93e2-9807e5ba6e54 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:28.611192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15167bff-2951-415e-8588-e7dcceafdc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:28.611816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15167bff-2951-415e-8588-e7dcceafdc55 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:28.612125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15167bff-2951-415e-8588-e7dcceafdc55 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:28.612396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15167bff-2951-415e-8588-e7dcceafdc55 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:28.619404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15167bff-2951-415e-8588-e7dcceafdc55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:28.620870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15167bff-2951-415e-8588-e7dcceafdc55 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-15167bff-2951-415e-8588-e7dcceafdc55 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:40:28.632719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:40:28.634101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:40:28.635716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 583/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:40:28 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:40:28.643321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:28.715195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:28.754087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dae13-344b-4c6c-bd61-a69a3c16d93d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:28.791329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649a115-1290-4380-85a7-d7be9bdc213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:28.791916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649a115-1290-4380-85a7-d7be9bdc213d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:28.792235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649a115-1290-4380-85a7-d7be9bdc213d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:28.792504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649a115-1290-4380-85a7-d7be9bdc213d None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:28.793153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649a115-1290-4380-85a7-d7be9bdc213d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:28.793452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649a115-1290-4380-85a7-d7be9bdc213d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:28.794828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649a115-1290-4380-85a7-d7be9bdc213d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:28.802158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 583/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Oct 4 08:40:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:30.914069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e4404-d3d3-485c-a423-674c62632064 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:30.914629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e4404-d3d3-485c-a423-674c62632064 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:30.914629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e4404-d3d3-485c-a423-674c62632064 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:30.916058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 584/1167] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:30.924831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4c514-b573-41f3-84a7-f685fe4dd97a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:30.925553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4c514-b573-41f3-84a7-f685fe4dd97a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:30.925749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4c514-b573-41f3-84a7-f685fe4dd97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:30.950930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-43b4c514-b573-41f3-84a7-f685fe4dd97a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:30.952151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-43b4c514-b573-41f3-84a7-f685fe4dd97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:31.026722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 584/1168] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:31.292909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e4404-d3d3-485c-a423-674c62632064 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:31.310196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:31.407297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e4404-d3d3-485c-a423-674c62632064 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:31.442226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47681b-fee3-411e-8fb8-61a2d5db1cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:31.443025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47681b-fee3-411e-8fb8-61a2d5db1cf6 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:31.443318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47681b-fee3-411e-8fb8-61a2d5db1cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:31.443709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47681b-fee3-411e-8fb8-61a2d5db1cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:31.444453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47681b-fee3-411e-8fb8-61a2d5db1cf6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:31.445806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47681b-fee3-411e-8fb8-61a2d5db1cf6 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:31.475156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 585/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Tue Oct 4 08:40:31 2022] POST /identity/v3/domains => generated 267 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:40:33.625905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e521d6d-14b1-4452-b9be-993fb863eb8f None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:33.626784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e521d6d-14b1-4452-b9be-993fb863eb8f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:33.626784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e521d6d-14b1-4452-b9be-993fb863eb8f None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:33.628972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 585/1170] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:33.634671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88c1e2-23d1-4c23-b31d-32d96978d00b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:33.635079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88c1e2-23d1-4c23-b31d-32d96978d00b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:33.635207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88c1e2-23d1-4c23-b31d-32d96978d00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:33.655024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3b88c1e2-23d1-4c23-b31d-32d96978d00b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:33.656266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3b88c1e2-23d1-4c23-b31d-32d96978d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:33.722160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:33.727391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 586/1171] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:34.004450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e521d6d-14b1-4452-b9be-993fb863eb8f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:34.070072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:34.115230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e521d6d-14b1-4452-b9be-993fb863eb8f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:34.149497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211600b-f447-4cad-b104-63a0c1be19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:34.150111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211600b-f447-4cad-b104-63a0c1be19bc None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:34.150384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211600b-f447-4cad-b104-63a0c1be19bc None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:34.150651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211600b-f447-4cad-b104-63a0c1be19bc None admin] PATH_INFO: `/v3/domains/40c7171670be4414bb5af6ebe619bc2f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:34.160939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211600b-f447-4cad-b104-63a0c1be19bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=40c7171670be4414bb5af6ebe619bc2f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:34.162380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211600b-f447-4cad-b104-63a0c1be19bc None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:34.163957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 586/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Tue Oct 4 08:40:34 2022] GET /identity/v3/domains/40c7171670be4414bb5af6ebe619bc2f => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:34.170090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88c1e2-23d1-4c23-b31d-32d96978d00b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:34.281771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88c1e2-23d1-4c23-b31d-32d96978d00b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:34.320597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a387765-f440-4107-b717-486fcf11a65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:34.321187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387765-f440-4107-b717-486fcf11a65c None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:34.321462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387765-f440-4107-b717-486fcf11a65c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:34.321729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387765-f440-4107-b717-486fcf11a65c None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:34.322432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a387765-f440-4107-b717-486fcf11a65c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:34.323931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a387765-f440-4107-b717-486fcf11a65c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:34.353706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 587/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:40:34 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:40:36.512589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c206b-4e05-4ab3-a4ca-ea2996fe6922 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:36.513125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c206b-4e05-4ab3-a4ca-ea2996fe6922 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:36.513125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c206b-4e05-4ab3-a4ca-ea2996fe6922 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:36.514461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 587/1174] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:36.521206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3d111-eae4-46d7-9ebc-6d56aae9ea55 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:36.521637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3d111-eae4-46d7-9ebc-6d56aae9ea55 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:36.521730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3d111-eae4-46d7-9ebc-6d56aae9ea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:36.544183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2ad3d111-eae4-46d7-9ebc-6d56aae9ea55 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:36.545353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2ad3d111-eae4-46d7-9ebc-6d56aae9ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:36.592404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:36.613832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 588/1175] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:36.881537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c206b-4e05-4ab3-a4ca-ea2996fe6922 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:37.000965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c206b-4e05-4ab3-a4ca-ea2996fe6922 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:37.021428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:37.037017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7db35-2b8d-4625-98c6-bea16d9f835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:37.037642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7db35-2b8d-4625-98c6-bea16d9f835a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:37.037916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7db35-2b8d-4625-98c6-bea16d9f835a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:37.038180 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7db35-2b8d-4625-98c6-bea16d9f835a None admin] PATH_INFO: `/v3/domains/40c7171670be4414bb5af6ebe619bc2f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:37.039342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7db35-2b8d-4625-98c6-bea16d9f835a None admin] RBAC: Authorizing `identity:get_domain(domain_id=40c7171670be4414bb5af6ebe619bc2f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:37.040797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7db35-2b8d-4625-98c6-bea16d9f835a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:37.042366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 588/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Tue Oct 4 08:40:36 2022] GET /identity/v3/domains/40c7171670be4414bb5af6ebe619bc2f => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:37.048426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3d111-eae4-46d7-9ebc-6d56aae9ea55 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:37.151149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3d111-eae4-46d7-9ebc-6d56aae9ea55 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:37.188441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b2413-ce8b-4ee4-a852-f57d82fbd9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:37.188989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2413-ce8b-4ee4-a852-f57d82fbd9b4 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:37.189268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2413-ce8b-4ee4-a852-f57d82fbd9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:37.189533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2413-ce8b-4ee4-a852-f57d82fbd9b4 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:37.190191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b2413-ce8b-4ee4-a852-f57d82fbd9b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:37.191619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b2413-ce8b-4ee4-a852-f57d82fbd9b4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:37.217040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 589/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:40:37 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:40:39.391408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d35da7-3896-41a8-8e52-93eba18e7fee None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:39.392362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d35da7-3896-41a8-8e52-93eba18e7fee None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:39.392362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d35da7-3896-41a8-8e52-93eba18e7fee None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:39.394489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 589/1178] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:39.400987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75ab19-a049-4a79-9915-766625579ee8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:39.401410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75ab19-a049-4a79-9915-766625579ee8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:39.401553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75ab19-a049-4a79-9915-766625579ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:39.429659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9e75ab19-a049-4a79-9915-766625579ee8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:39.430850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9e75ab19-a049-4a79-9915-766625579ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:39.461181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:39.502979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 590/1179] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:39.770980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d35da7-3896-41a8-8e52-93eba18e7fee None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:39.885882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d35da7-3896-41a8-8e52-93eba18e7fee None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:39.932511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:39.933134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:39.933409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:39.933677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:39.944748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:40:39.949520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:40:39.950622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:40:39.951563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 590/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:40:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:40:39.957626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75ab19-a049-4a79-9915-766625579ee8 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:40.064672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75ab19-a049-4a79-9915-766625579ee8 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:40.099832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:40.100451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:40.100734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:40.101010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:40.101669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:40.101969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:40.103331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:40.111581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 591/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:40:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:40:40.118444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:40.180607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:40.243341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e9e5e-499b-42f1-bcb8-a91d938c07f1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:40.280835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:40.281466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:40.281734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:40.281999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] PATH_INFO: `/v3/users/71d5258f32304865a34d5e9bcd2d308f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:40.296338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] RBAC: Authorizing `identity:get_user(user_id=71d5258f32304865a34d5e9bcd2d308f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:40.297766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:40.299362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 591/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:40 2022] GET /identity/v3/users/71d5258f32304865a34d5e9bcd2d308f => generated 311 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:40.305610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:40.337445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:40.408442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918145e-eacb-4f78-8d52-d28aa0e45c20 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:40.446748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:40.447349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:40.447654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:40.447932 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:40.516413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:40.517927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:40.547413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:40.622921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 592/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:40:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:40:40.629059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:40.744183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b63c2-7698-43b9-8582-4acb10e252e1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:40.751544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:40.780681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebed5b-428e-45d5-b786-7e61f211f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:40.781325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebed5b-428e-45d5-b786-7e61f211f63c None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:40.781549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebed5b-428e-45d5-b786-7e61f211f63c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:40.781812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebed5b-428e-45d5-b786-7e61f211f63c None admin] PATH_INFO: `/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:40.782953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebed5b-428e-45d5-b786-7e61f211f63c None admin] RBAC: Authorizing `identity:get_project(project_id=f5c1f32f7a7444068500e2fbdc3099b5)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:40.784384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebed5b-428e-45d5-b786-7e61f211f63c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:40.785999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 592/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:40 2022] GET /identity/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:40.792309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:40.898023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e075772-5538-4fb7-b998-29230139bd7c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:40.927088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:40.933867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b9174-ea11-492d-9ec8-c99ab320c35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:40.934452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9174-ea11-492d-9ec8-c99ab320c35d None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:40.934728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9174-ea11-492d-9ec8-c99ab320c35d None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:40.934996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9174-ea11-492d-9ec8-c99ab320c35d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:40.935703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9174-ea11-492d-9ec8-c99ab320c35d None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=f5c1f32f7a7444068500e2fbdc3099b5, user.id=71d5258f32304865a34d5e9bcd2d308f) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:40.936038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9174-ea11-492d-9ec8-c99ab320c35d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:40.937452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9174-ea11-492d-9ec8-c99ab320c35d None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:40.942940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 593/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:40:40 2022] GET /identity/v3/role_assignments?user.id=71d5258f32304865a34d5e9bcd2d308f&scope.project.id=f5c1f32f7a7444068500e2fbdc3099b5&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:43.051106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af72f-96ab-402c-bcdb-daf64c7fb42f None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:43.051996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af72f-96ab-402c-bcdb-daf64c7fb42f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:43.052200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af72f-96ab-402c-bcdb-daf64c7fb42f None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:43.054276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 593/1186] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:43.060453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a7fd-5ad8-40cb-b2ba-6c26f692bb76 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:43.061209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a7fd-5ad8-40cb-b2ba-6c26f692bb76 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:43.061349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a7fd-5ad8-40cb-b2ba-6c26f692bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:43.088864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-fd51a7fd-5ad8-40cb-b2ba-6c26f692bb76 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:43.090062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fd51a7fd-5ad8-40cb-b2ba-6c26f692bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:43.160076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 594/1187] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:43.423864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af72f-96ab-402c-bcdb-daf64c7fb42f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:43.541203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af72f-96ab-402c-bcdb-daf64c7fb42f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:43.584088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:43.584998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:43.585434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:43.585853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:43.601774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:40:43.606559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:40:43.610220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:40:43.611325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 594/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:40:43.614246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51a7fd-5ad8-40cb-b2ba-6c26f692bb76 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:43.725793 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51a7fd-5ad8-40cb-b2ba-6c26f692bb76 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:43.750703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:43.770028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:43.770984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:43.771460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:43.771795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:43.772499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:43.772805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:43.774159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:43.782442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 595/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:43.788494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:43.825252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:43.893890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084c4c3-943c-4c3f-a070-502a00edecc6 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:43.929896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:43.930513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:43.930786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:43.931055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] PATH_INFO: `/v3/users/71d5258f32304865a34d5e9bcd2d308f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:43.932350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] RBAC: Authorizing `identity:get_user(user_id=71d5258f32304865a34d5e9bcd2d308f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:43.933786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:43.935374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 595/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:43 2022] GET /identity/v3/users/71d5258f32304865a34d5e9bcd2d308f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:43.942199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:44.050017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35e9f0-243f-41f5-8940-cd35f91bbe15 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:44.084810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8daee-4e24-4709-8d28-632fa4369f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:44.085376 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8daee-4e24-4709-8d28-632fa4369f1c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:44.085675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8daee-4e24-4709-8d28-632fa4369f1c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:44.085940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8daee-4e24-4709-8d28-632fa4369f1c None admin] PATH_INFO: `/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:44.087158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8daee-4e24-4709-8d28-632fa4369f1c None admin] RBAC: Authorizing `identity:get_project(project_id=f5c1f32f7a7444068500e2fbdc3099b5)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:44.088596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8daee-4e24-4709-8d28-632fa4369f1c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:44.090214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 596/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:43 2022] GET /identity/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:44.096664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:44.200168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ddedf-e829-4418-adfc-5b3e7817db73 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:44.201799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:44.240849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2096d26-fd45-472e-9381-eb4c0066843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:44.241501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2096d26-fd45-472e-9381-eb4c0066843c None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:44.241781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2096d26-fd45-472e-9381-eb4c0066843c None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:44.242050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2096d26-fd45-472e-9381-eb4c0066843c None admin] PATH_INFO: `/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5/users/71d5258f32304865a34d5e9bcd2d308f/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:44.244254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2096d26-fd45-472e-9381-eb4c0066843c None admin] RBAC: Authorizing `identity:create_grant(project_id=f5c1f32f7a7444068500e2fbdc3099b5, user_id=71d5258f32304865a34d5e9bcd2d308f, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:44.245694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2096d26-fd45-472e-9381-eb4c0066843c None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:44.261331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 596/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:40:44 2022] PUT /identity/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5/users/71d5258f32304865a34d5e9bcd2d308f/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:40:46.342257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908afa6-409e-43bb-8111-0cb15265a228 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:46.342847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908afa6-409e-43bb-8111-0cb15265a228 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:46.342847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908afa6-409e-43bb-8111-0cb15265a228 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:46.344163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 597/1193] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:46.350609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3515f-b0d2-4cf7-8169-8c25317de2f4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:46.351286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3515f-b0d2-4cf7-8169-8c25317de2f4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:46.351474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3515f-b0d2-4cf7-8169-8c25317de2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:46.381929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9df3515f-b0d2-4cf7-8169-8c25317de2f4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:46.383117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9df3515f-b0d2-4cf7-8169-8c25317de2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:46.461742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 597/1194] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:40:46.747267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908afa6-409e-43bb-8111-0cb15265a228 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:46.815259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:46.861891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908afa6-409e-43bb-8111-0cb15265a228 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:46.907413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:46.908078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:46.908354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:46.908627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:46.920751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:40:46.925509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:40:46.926782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:40:46.927991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 598/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:40:46.933544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3515f-b0d2-4cf7-8169-8c25317de2f4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:47.039863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:47.050772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3515f-b0d2-4cf7-8169-8c25317de2f4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:47.083777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:47.084405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.084680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.084947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.085606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:47.085905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:47.087268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:47.094844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 598/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:40:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:47.102431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:47.204640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758066-61b8-4a4e-b6da-aab2b3cdb1ad None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:47.219768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:47.241310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:47.241919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.242191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.242451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] PATH_INFO: `/v3/users/71d5258f32304865a34d5e9bcd2d308f` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.243673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] RBAC: Authorizing `identity:get_user(user_id=71d5258f32304865a34d5e9bcd2d308f)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:47.245117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:47.246652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 599/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Oct 4 08:40:47 2022] GET /identity/v3/users/71d5258f32304865a34d5e9bcd2d308f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:47.252941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:47.358286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a500a-60fd-4c52-9e7f-23fb822c9836 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:47.392893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:47.393480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.393754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.394021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.417691 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:47.462021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:47.463447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:47.565623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 599/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:40:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:40:47.572612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:47.682590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2819f-4699-402e-be80-6b706f330db8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:47.717754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322619bf-4fd6-4e24-a917-c581595eca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:47.718368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322619bf-4fd6-4e24-a917-c581595eca00 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.718641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322619bf-4fd6-4e24-a917-c581595eca00 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.718915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322619bf-4fd6-4e24-a917-c581595eca00 None admin] PATH_INFO: `/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.720096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322619bf-4fd6-4e24-a917-c581595eca00 None admin] RBAC: Authorizing `identity:get_project(project_id=f5c1f32f7a7444068500e2fbdc3099b5)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:47.721520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322619bf-4fd6-4e24-a917-c581595eca00 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:47.723150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 600/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Oct 4 08:40:47 2022] GET /identity/v3/projects/f5c1f32f7a7444068500e2fbdc3099b5 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:47.732865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:47.792484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:47.841226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1d686-ed2e-4647-b089-445b3d21a242 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:47.879865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24997784-0519-4831-99ec-192ec6433757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:47.880480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997784-0519-4831-99ec-192ec6433757 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.880754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997784-0519-4831-99ec-192ec6433757 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.881028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997784-0519-4831-99ec-192ec6433757 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.881747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24997784-0519-4831-99ec-192ec6433757 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=f5c1f32f7a7444068500e2fbdc3099b5, user.id=71d5258f32304865a34d5e9bcd2d308f) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:40:47.882053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24997784-0519-4831-99ec-192ec6433757 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:47.883487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24997784-0519-4831-99ec-192ec6433757 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:47.889350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 600/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:40:47 2022] GET /identity/v3/role_assignments?user.id=71d5258f32304865a34d5e9bcd2d308f&scope.project.id=f5c1f32f7a7444068500e2fbdc3099b5&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:40:57.712871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=75 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:57.886685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=41 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:43.364580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35d26-3ef2-4722-867f-71c24181e6b1 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:43.364580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35d26-3ef2-4722-867f-71c24181e6b1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:43.364580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35d26-3ef2-4722-867f-71c24181e6b1 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:43.367056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 601/1201] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:41:43.373029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff5998-426f-4b2f-9637-6327ed7504ca None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:43.373851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff5998-426f-4b2f-9637-6327ed7504ca None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:43.374610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff5998-426f-4b2f-9637-6327ed7504ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:43.400637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-52ff5998-426f-4b2f-9637-6327ed7504ca None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:41:43.402017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-52ff5998-426f-4b2f-9637-6327ed7504ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:41:43.496439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 601/1202] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:41:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:41:43.796968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c35d26-3ef2-4722-867f-71c24181e6b1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:43.936286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c35d26-3ef2-4722-867f-71c24181e6b1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:43.984854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:43.985488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:43.985761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:43.986032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:43.998289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:44.000069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:41:44.018450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:44.019531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:41:44.020941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:41:44.020941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:44.020941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:41:44.020941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 602/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:44.026188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff5998-426f-4b2f-9637-6327ed7504ca None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:44.160432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff5998-426f-4b2f-9637-6327ed7504ca None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:44.196417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:44.203297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242e126-6342-4d3f-9be9-4b3399e8bf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:44.203952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e126-6342-4d3f-9be9-4b3399e8bf02 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:44.204475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e126-6342-4d3f-9be9-4b3399e8bf02 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:44.204804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242e126-6342-4d3f-9be9-4b3399e8bf02 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:44.205453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242e126-6342-4d3f-9be9-4b3399e8bf02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:44.205759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242e126-6342-4d3f-9be9-4b3399e8bf02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:44.207136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242e126-6342-4d3f-9be9-4b3399e8bf02 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:44.220493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 602/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:44.226204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:44.302489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:44.367731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e71ee4-0eca-4b3a-96ab-663eee8e84f3 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:44.407918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943da1ae-561a-42de-bc56-9674d6dc0486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:44.408564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da1ae-561a-42de-bc56-9674d6dc0486 None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:44.408864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da1ae-561a-42de-bc56-9674d6dc0486 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:44.409145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da1ae-561a-42de-bc56-9674d6dc0486 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:44.409800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943da1ae-561a-42de-bc56-9674d6dc0486 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:44.411349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943da1ae-561a-42de-bc56-9674d6dc0486 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:44.438017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 603/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Oct 4 08:41:44 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:41:46.970855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6e949-0aca-4630-8bee-e49b9a8640ea None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:46.971403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6e949-0aca-4630-8bee-e49b9a8640ea None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:46.971403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6e949-0aca-4630-8bee-e49b9a8640ea None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:46.972887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 603/1206] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:41:46.978552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee1514-626f-4c65-b3aa-e3593814f017 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:46.979001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee1514-626f-4c65-b3aa-e3593814f017 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:46.979106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee1514-626f-4c65-b3aa-e3593814f017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:46.998882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-97ee1514-626f-4c65-b3aa-e3593814f017 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:41:47.000433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-97ee1514-626f-4c65-b3aa-e3593814f017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:41:47.076538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 604/1207] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:41:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:41:47.341908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6e949-0aca-4630-8bee-e49b9a8640ea None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:47.458819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6e949-0aca-4630-8bee-e49b9a8640ea None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:47.497523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:47.498166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:47.498430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:47.498728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:47.511451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:41:47.516990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:47.518015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:47.518908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 604/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:47.525103 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee1514-626f-4c65-b3aa-e3593814f017 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:47.568399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:47.640344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee1514-626f-4c65-b3aa-e3593814f017 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:47.679044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:47.679682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:47.679969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:47.680281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:47.680911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:47.681328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:47.682615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:47.691052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 605/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:41:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:47.697277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:47.755286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:47.827639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333adfc5-7cca-4e01-8cec-54ae585293e1 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:47.870074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:47.870708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:47.870984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:47.871394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:47.883470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:47.884996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:41:47.901848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:47.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:41:47.904266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:41:47.904266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:47.904266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:47.904266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 605/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:47.910607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:48.021954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:48.025011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261308f9-0cc9-4795-868e-6cf280e77ce4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:48.059546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:48.060210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:48.060474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:48.060741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:48.061442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:48.061757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:48.063199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:48.080513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 606/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:48.081802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:48.188868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d8fb2-38df-49b3-a8f3-6547e80b22e5 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:48.190295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:48.226738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:48.227408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:48.227707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:48.228036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:48.237291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:48.239557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:41:48.252946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:48.253970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:48.259820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 606/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:48.260502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:48.369641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0ac6d-061f-4bd5-98c5-032b083f2430 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:48.405995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:48.406913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:48.407381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:48.407794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:48.408807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:48.409143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:48.411194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:48.433705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 607/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Oct 4 08:41:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:48.441112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:48.548126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bf3aa-90b3-4441-88d3-94a559285f85 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:48.584384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:48.585317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:48.585768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:48.586209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:48.618970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:48.663758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:48.665486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:48.785809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 607/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:41:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:41:48.792108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:48.816441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:48.928446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc27b8-6ec4-48fe-926c-61e495a3fdb4 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:48.987985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:48.988729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:48.989096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:48.989615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:49.007241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:49.008818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:41:49.025617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:49.027251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:41:49.028276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:41:49.028276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:49.028276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:41:49.028276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 608/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:49.032844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:49.094591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:49.135867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70fa0a-c9cc-4839-bf73-991240b218a0 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:49.173643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:49.174356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:49.174699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:49.174966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:49.175618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:49.175950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:49.177453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:49.189160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 608/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:49.195031 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:49.270278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:49.303232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9659-bbda-44e8-92ce-9b5a3a7713b8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:49.338909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:49.339675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:49.340059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:49.340330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:49.348294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:49.349877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:41:49.362810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:49.366852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:41:49.368244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 609/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:49.370147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:49.474889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d1bcb-3749-4199-8eb3-fbfc12e0b165 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:49.500074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:49.513980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcc968-91d7-4df2-bf29-492e635fb914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:49.514708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcc968-91d7-4df2-bf29-492e635fb914 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:49.515097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcc968-91d7-4df2-bf29-492e635fb914 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:49.515554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcc968-91d7-4df2-bf29-492e635fb914 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:49.516818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcc968-91d7-4df2-bf29-492e635fb914 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:49.517328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcc968-91d7-4df2-bf29-492e635fb914 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:49.519677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcc968-91d7-4df2-bf29-492e635fb914 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:49.537325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 609/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:49.543172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:49.655673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfe9bf-2f17-49ab-bf82-2124feac6dcf None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:49.687226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:49.694122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450528e4-39c5-463a-8e8a-2881e26b2e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:49.694770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450528e4-39c5-463a-8e8a-2881e26b2e55 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:49.694994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450528e4-39c5-463a-8e8a-2881e26b2e55 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:49.695289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450528e4-39c5-463a-8e8a-2881e26b2e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:49.696096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450528e4-39c5-463a-8e8a-2881e26b2e55 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=9cee970f95754740b2f19e64f4653c7f) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:49.696401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450528e4-39c5-463a-8e8a-2881e26b2e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:49.697879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450528e4-39c5-463a-8e8a-2881e26b2e55 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:49.703639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 610/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:41:49 2022] GET /identity/v3/role_assignments?user.id=9cee970f95754740b2f19e64f4653c7f&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:41:51.842037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a9778-e536-4b26-a925-3ab181e4ca19 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:51.842653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a9778-e536-4b26-a925-3ab181e4ca19 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:51.842653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a9778-e536-4b26-a925-3ab181e4ca19 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:51.844475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 610/1220] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:41:51.850366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074898-7017-4c7f-848f-8c36a0ee90ff None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:51.850915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074898-7017-4c7f-848f-8c36a0ee90ff None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:51.850915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074898-7017-4c7f-848f-8c36a0ee90ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:51.870435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-71074898-7017-4c7f-848f-8c36a0ee90ff None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:41:51.871779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-71074898-7017-4c7f-848f-8c36a0ee90ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:41:51.947135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 611/1221] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:41:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:41:52.209210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a9778-e536-4b26-a925-3ab181e4ca19 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:52.316386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a9778-e536-4b26-a925-3ab181e4ca19 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:52.353595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:52.354221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:52.354488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:52.354750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:52.367163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:41:52.373884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:52.375475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:52.376465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 611/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:41:52.383281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71074898-7017-4c7f-848f-8c36a0ee90ff None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:52.500428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71074898-7017-4c7f-848f-8c36a0ee90ff None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:52.527595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:52.538954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:52.539530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:52.539837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:52.540294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:52.541055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] RBAC: Adding query filter params (name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:52.541429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:52.542877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:52.552318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 612/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:41:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:41:52.558864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:52.607784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:52.665963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3268b8-1427-4dbc-bfd7-bbc8d8a5f837 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:52.700745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:52.701375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:52.701652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:52.701919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:52.712343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:52.713868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:41:52.729946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:52.736481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:41:52.737832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:41:52.737832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:52.737832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:52.737832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 612/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:52.738153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:52.846544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44432855-7dd0-4c0a-a5ef-9c774456db62 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:52.884894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:52.885628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:52.885724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:52.885977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:52.886637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:52.886965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:52.888572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:52.902249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 613/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:41:52.908141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:53.017534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:53.024466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb01e9-5598-4eb2-a92e-77e24928ccf3 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:53.064233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:53.065289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:53.065793 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:53.066143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:53.075731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:53.077296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:41:53.092688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:53.094168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:53.095391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 613/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:53.101173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:53.159851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:53.224560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e84b27-5b96-4d01-89e3-46eec198fbb7 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:53.268094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:53.268735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:53.269043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:53.269312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:53.270268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:53.270661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:53.272386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:53.293220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 614/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Oct 4 08:41:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:41:53.299318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:53.416880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3325b2-6e22-4d81-822f-08f86068c774 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:53.458839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:53.459824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:53.460153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:53.460422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:53.472856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:53.474482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:41:53.490623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:53.491984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:41:53.492966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:41:53.492966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:53.492966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:53.492966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 614/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:53.497895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:53.610060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194433c3-92b1-48e2-a35b-8234b526e1de None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:53.613146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:53.651279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:53.651922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:53.652203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:53.652485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:53.653160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:53.653454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:53.655013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:53.669619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 615/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:41:53.676536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:53.688863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:53.790014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e3891-023f-4aed-bf52-691f0f05c6b7 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:53.829301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:53.830120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:53.830409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:53.830697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:53.838972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:53.841167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:41:53.856399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:53.857539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:53.858606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 615/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:53.864439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:53.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b9e2a-1ec0-4da5-96c5-90192590fc25 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:54.012755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf4d94-a6a1-4487-b1bf-0a985d9ae813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:54.013361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf4d94-a6a1-4487-b1bf-0a985d9ae813 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:54.013639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf4d94-a6a1-4487-b1bf-0a985d9ae813 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:54.013904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf4d94-a6a1-4487-b1bf-0a985d9ae813 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:54.014592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf4d94-a6a1-4487-b1bf-0a985d9ae813 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:54.014898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf4d94-a6a1-4487-b1bf-0a985d9ae813 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:54.016400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf4d94-a6a1-4487-b1bf-0a985d9ae813 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:54.028531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 616/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:54.035457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:54.102835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:54.141931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a0ef4-16c6-4eb9-86c4-40edc5a7a117 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:54.177785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569883ea-7ea3-4266-99d1-cf2e21682927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:54.178445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569883ea-7ea3-4266-99d1-cf2e21682927 None admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:54.178719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569883ea-7ea3-4266-99d1-cf2e21682927 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:54.179037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569883ea-7ea3-4266-99d1-cf2e21682927 None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/9cee970f95754740b2f19e64f4653c7f/roles/4460b00f7dab40cd8eb26f0b606b4101` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:54.193988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569883ea-7ea3-4266-99d1-cf2e21682927 None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=9cee970f95754740b2f19e64f4653c7f, role_id=4460b00f7dab40cd8eb26f0b606b4101)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:54.195421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569883ea-7ea3-4266-99d1-cf2e21682927 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:54.208954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 616/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:41:54 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/9cee970f95754740b2f19e64f4653c7f/roles/4460b00f7dab40cd8eb26f0b606b4101 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:41:56.451288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e86b-417f-45e5-a6a1-2b5e381a399b None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:56.451911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e86b-417f-45e5-a6a1-2b5e381a399b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:56.451911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e86b-417f-45e5-a6a1-2b5e381a399b None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:56.453339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 617/1233] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:41:56.459361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53050ee5-da4c-40b8-9398-bcfda28677df None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:56.459854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53050ee5-da4c-40b8-9398-bcfda28677df None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:56.459989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53050ee5-da4c-40b8-9398-bcfda28677df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:56.479358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-53050ee5-da4c-40b8-9398-bcfda28677df None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:41:56.480611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-53050ee5-da4c-40b8-9398-bcfda28677df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:41:56.553539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 617/1234] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:41:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:41:56.847461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155e86b-417f-45e5-a6a1-2b5e381a399b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:56.875621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:56.960254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155e86b-417f-45e5-a6a1-2b5e381a399b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:57.000805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:57.001438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:57.001707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:57.001973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:57.013259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:41:57.018530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:57.019792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:41:57.020840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 618/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Oct 4 08:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:57.026467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53050ee5-da4c-40b8-9398-bcfda28677df None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:57.079064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:57.146370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53050ee5-da4c-40b8-9398-bcfda28677df None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:57.191578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:57.192230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:57.192550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:57.192814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:57.193485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] RBAC: Adding query filter params (name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:57.193784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:57.195153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:57.203494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 618/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Oct 4 08:41:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:57.211182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:57.295945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:57.311711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009587db-92fc-4df5-9830-1f8405288808 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:57.352879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:57.353539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:57.353877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:57.354182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:57.365653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:57.367220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:41:57.383525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:57.384727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:41:57.385673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:41:57.385673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:57.385673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:41:57.385673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 619/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:57.390764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:57.507356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441a0d5-e0f1-42e4-bb5f-d8feb615b868 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:57.518866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:57.547018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:57.547598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:57.547976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:57.548254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:57.548902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:57.549224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:57.550627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:57.564817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 619/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:57.571054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:57.683034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7ffc2-173b-4069-94e8-87d49f6513cb None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:57.725492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:57.726429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:57.726765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:57.727012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:57.730250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:57.736362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:57.738226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:41:57.750669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:57.751830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:41:57.757962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 620/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:57.758601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:57.879239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571ae2-e9e5-4445-9d04-4650448cdb14 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:57.918381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:57.919060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:57.919394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:57.919665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:57.920445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:57.920760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:57.922219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:57.940981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 620/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Oct 4 08:41:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:57.948661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:58.070634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dca0d-2636-42cb-8b7f-82c54095a872 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:58.108397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:58.109010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:58.109302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:58.109560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:58.174946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:58.183016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:58.184735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:58.288882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 621/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:41:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:41:58.294964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:58.322960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:58.403063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea588d6-4672-4a4f-a062-ce89cc194172 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:58.440355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:58.440996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:58.441382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:58.441748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:58.455866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:58.457657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:41:58.475557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:58.476825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:41:58.477927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:41:58.477927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:41:58.477927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:58.477927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 621/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:58.482717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:58.584389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:58.590902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4713c6a-3efb-4be9-8f06-d3e7b6ddd57c None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:58.635865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:58.636803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:58.637220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:58.637649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:58.638514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:58.638928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:58.640598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:58.653953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 622/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:58.659936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:58.744867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:58.772622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3f5b-69dc-4eb9-8390-5df924b8f5df None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:58.812316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:58.812991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:58.813337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:58.813607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:58.821750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:58.823423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:41:58.837791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:41:58.839051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:41:58.840297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 622/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:41:58.851104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:58.974792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb602a-4db5-4849-96ee-87dde2ac9ee1 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:59.015235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b89617-f702-40aa-b4c8-f2ac68970bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:59.015864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b89617-f702-40aa-b4c8-f2ac68970bb1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:59.016208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b89617-f702-40aa-b4c8-f2ac68970bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:59.016485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b89617-f702-40aa-b4c8-f2ac68970bb1 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:59.017188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b89617-f702-40aa-b4c8-f2ac68970bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:59.017494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b89617-f702-40aa-b4c8-f2ac68970bb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:59.020435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b89617-f702-40aa-b4c8-f2ac68970bb1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:59.033348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 623/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:41:59.040449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:41:59.151819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0818b-b79f-48e4-a529-c4934fcff902 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:41:59.188634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5e8ef-5794-46b9-94c2-b21c14edb7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:41:59.189285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e8ef-5794-46b9-94c2-b21c14edb7e7 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:41:59.189563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e8ef-5794-46b9-94c2-b21c14edb7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:41:59.189830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5e8ef-5794-46b9-94c2-b21c14edb7e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:41:59.190551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5e8ef-5794-46b9-94c2-b21c14edb7e7 None admin] RBAC: Adding query filter params (role.id=4460b00f7dab40cd8eb26f0b606b4101, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=9cee970f95754740b2f19e64f4653c7f) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:41:59.190851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5e8ef-5794-46b9-94c2-b21c14edb7e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:41:59.192331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5e8ef-5794-46b9-94c2-b21c14edb7e7 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:41:59.198805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 623/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:41:59 2022] GET /identity/v3/role_assignments?user.id=9cee970f95754740b2f19e64f4653c7f&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=4460b00f7dab40cd8eb26f0b606b4101 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:42:01.688034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3ce4-9407-41e0-b4c0-c22e1024e7f4 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:01.688685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3ce4-9407-41e0-b4c0-c22e1024e7f4 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:01.688685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3ce4-9407-41e0-b4c0-c22e1024e7f4 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:01.690617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 624/1247] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:42:01.696712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00cbe-1da2-440e-b326-b201cf116bcf None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:01.697159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00cbe-1da2-440e-b326-b201cf116bcf None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:01.697242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00cbe-1da2-440e-b326-b201cf116bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:01.703764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:01.717015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c0a00cbe-1da2-440e-b326-b201cf116bcf None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:01.718221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c0a00cbe-1da2-440e-b326-b201cf116bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:01.795412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 624/1248] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:42:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:42:02.063074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f3ce4-9407-41e0-b4c0-c22e1024e7f4 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:02.093470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:02.187741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f3ce4-9407-41e0-b4c0-c22e1024e7f4 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:02.224305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:02.224932 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:02.225202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:02.225679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:02.238477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:42:02.244360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:02.250782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:02.251867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 625/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:02.252793 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00cbe-1da2-440e-b326-b201cf116bcf None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:02.356133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00cbe-1da2-440e-b326-b201cf116bcf None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:02.386243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:02.392372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:02.393019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:02.393302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:02.393573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:02.394237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:02.394536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:02.396003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:02.410077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 625/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:42:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:02.411347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:02.498323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:02.524654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde144-902a-41de-940b-f34b75e4d914 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:02.561338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:02.562098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:02.562413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:02.562685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:02.574280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:02.575935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:42:02.596392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:02.597791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:42:02.599004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:42:02.599004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:02.599004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:02.599004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 626/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:02.604391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:02.718885 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74f0d6-cbc5-4c8b-9181-bca163af1737 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:02.754059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:02.754660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:02.754907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:02.755329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:02.756103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:02.756522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:02.757938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:02.771452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 626/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:02.777650 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:02.883468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf480cf-54f5-4b63-9fe3-24d9fe53ba96 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:02.912200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:02.922716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:02.923516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:02.923804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:02.924189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:02.933441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:02.934986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:42:02.948088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:02.949337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:02.950380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 627/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:02.955176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:03.021659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:03.074066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47416f-ff09-4825-8844-de74eae50fcc None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:03.113708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:03.114404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:03.114749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:03.115024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:03.115801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:03.116142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:03.117682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:03.138537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 627/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Oct 4 08:42:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:03.144904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:03.253580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b2d12-4313-4dcf-a68b-7a8a773bf6a8 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:03.290997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:03.291666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:03.291879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:03.292211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:03.352498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:03.366218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:03.367685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:03.468174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 628/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:42:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:42:03.474454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:03.574068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867f30-5354-4e70-9ca0-24bc2cd14c4b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:03.589313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:03.613315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:03.614002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:03.614280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:03.614614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:03.628280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:03.630016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:42:03.648779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:03.649997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:42:03.651286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:42:03.651286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:03.651286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:03.651286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 628/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:03.655683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:03.744584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:03.758520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e68b8-8a95-442a-80a8-8346ad1d7a8b None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:03.793439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:03.794020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:03.794291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:03.794555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:03.795194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:03.795494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:03.796881 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:03.807895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 629/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:03.814695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:03.922782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154880-7784-4be6-b5ce-cd4ba7a8decd None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:03.957336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:03.957957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:03.958309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:03.958545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:03.965321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:03.966767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:42:03.978301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:03.979356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:03.980462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 629/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:03.985612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:04.091050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1edf79-aed3-4d92-bfeb-55327a6ecd3f None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:04.135309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960f4b3-7e05-4a3e-a1dd-869d18f47644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:04.136304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960f4b3-7e05-4a3e-a1dd-869d18f47644 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:04.136588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960f4b3-7e05-4a3e-a1dd-869d18f47644 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:04.136846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960f4b3-7e05-4a3e-a1dd-869d18f47644 None admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:04.137565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960f4b3-7e05-4a3e-a1dd-869d18f47644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:04.137868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960f4b3-7e05-4a3e-a1dd-869d18f47644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:04.139297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960f4b3-7e05-4a3e-a1dd-869d18f47644 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:04.152866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 630/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:04.162472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:04.192488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:04.277992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38c04-d6a1-4da8-b041-9a66efef2fdf None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:04.318392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a4495-64df-485d-9eca-17601fb416ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:04.318987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a4495-64df-485d-9eca-17601fb416ea None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:04.319262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a4495-64df-485d-9eca-17601fb416ea None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:04.319533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a4495-64df-485d-9eca-17601fb416ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:04.320277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a4495-64df-485d-9eca-17601fb416ea None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=9cee970f95754740b2f19e64f4653c7f) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:04.320584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a4495-64df-485d-9eca-17601fb416ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:04.322025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a4495-64df-485d-9eca-17601fb416ea None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:04.327263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 630/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:42:04 2022] GET /identity/v3/role_assignments?user.id=9cee970f95754740b2f19e64f4653c7f&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:06.473791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41217f85-2c11-44ab-b35c-0f4e85a16ffd None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:06.474820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41217f85-2c11-44ab-b35c-0f4e85a16ffd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:06.474820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41217f85-2c11-44ab-b35c-0f4e85a16ffd None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:06.477080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 631/1261] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:42:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:06.483253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44445e5-d779-4007-961e-7d4319cce0af None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:06.484009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44445e5-d779-4007-961e-7d4319cce0af None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:06.484160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44445e5-d779-4007-961e-7d4319cce0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:06.510746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e44445e5-d779-4007-961e-7d4319cce0af None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:06.511987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e44445e5-d779-4007-961e-7d4319cce0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:06.583613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 631/1262] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:42:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:42:06.870158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41217f85-2c11-44ab-b35c-0f4e85a16ffd None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:06.883385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:06.985795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41217f85-2c11-44ab-b35c-0f4e85a16ffd None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.020469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.021140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.021415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.021684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.036949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:42:07.041708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:07.042964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:07.044269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 632/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:07.049103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44445e5-d779-4007-961e-7d4319cce0af None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.070473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.154064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44445e5-d779-4007-961e-7d4319cce0af None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.188774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.189356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.189634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.189890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.190579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:07.190874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.192268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.200288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 632/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:07.209500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.289099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.325848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec1f5-a1ed-49a2-9e82-95c8dc963e67 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.360938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.361557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.361830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.362095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.372307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.373740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:42:07.388900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:07.389949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:42:07.390960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:42:07.390960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:07.390960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:07.390960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 633/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:07.398143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.496976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.526046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b16afa-ff7d-4de8-af4a-bc9dae1262d2 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.560754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.561339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.561609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.561877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.562492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:07.562799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.564184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.575794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 633/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:07.585201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.709752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaeabe-8ae4-4a3d-98bc-a6c6854ea028 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.729656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.749029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.749584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.750181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.750473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.759443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.760903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:42:07.772575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:07.773648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:07.774784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 634/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:07.781677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.884103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7658-4e6c-48c9-bbd4-a115fc3abd77 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.921570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.922165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.922443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.922714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.923430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:07.923734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.925200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.933652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.945644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 634/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:07.951660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:08.055502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf011327-ffbf-4a07-b7d1-51e45d1ccb49 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.097423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.098048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.098327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.098598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.109404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.110834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:42:08.125455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:08.126578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:42:08.128437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:42:08.128437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:08.128437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:08.128437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 635/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:08.136634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:08.254113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21366fba-7a80-4727-ad6e-76469b5da06a None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.289593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.290185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.290462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.290732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.291371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:08.291673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.293092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:08.304228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 635/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:08.310092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:08.343606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:08.418325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66342e2-8b32-45d3-8e5d-c05fae2060e3 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.453531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.454145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.454422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.454697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.461644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.463074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:42:08.474440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:08.475584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:08.477279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 636/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:08.484907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:08.566166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:08.624695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e4ff9-c6b7-4a82-a75c-1f90f56a6296 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.658920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77e147-d502-47bf-9e14-7d970a6a24c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.659530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77e147-d502-47bf-9e14-7d970a6a24c3 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.659808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77e147-d502-47bf-9e14-7d970a6a24c3 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.660105 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77e147-d502-47bf-9e14-7d970a6a24c3 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.660802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77e147-d502-47bf-9e14-7d970a6a24c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:08.661107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77e147-d502-47bf-9e14-7d970a6a24c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.662603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77e147-d502-47bf-9e14-7d970a6a24c3 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:08.674054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 636/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:42:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:08.683291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:08.779064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:08.790603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f762e-853e-4bdd-b6ae-458cc133bc22 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.825968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd59788-b356-4dd0-8f51-0f00e5c8da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.826543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd59788-b356-4dd0-8f51-0f00e5c8da0e None admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.826821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd59788-b356-4dd0-8f51-0f00e5c8da0e None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.827083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd59788-b356-4dd0-8f51-0f00e5c8da0e None admin] PATH_INFO: `/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/9cee970f95754740b2f19e64f4653c7f/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.829007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd59788-b356-4dd0-8f51-0f00e5c8da0e None admin] RBAC: Authorizing `identity:create_grant(project_id=c88e95e15cef4d41976389c691f4bca4, user_id=9cee970f95754740b2f19e64f4653c7f, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.830399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd59788-b356-4dd0-8f51-0f00e5c8da0e None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:08.846203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 637/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Oct 4 08:42:08 2022] PUT /identity/v3/projects/c88e95e15cef4d41976389c691f4bca4/users/9cee970f95754740b2f19e64f4653c7f/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:42:10.954381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d15cc8-2704-4d2e-93bd-fd95ae0aacc8 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:10.955267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d15cc8-2704-4d2e-93bd-fd95ae0aacc8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:10.955267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d15cc8-2704-4d2e-93bd-fd95ae0aacc8 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:10.957513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 637/1274] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:42:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:10.966714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4b2ed-6d0b-451b-bd3e-0c7bdfa367ad None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:10.967144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4b2ed-6d0b-451b-bd3e-0c7bdfa367ad None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:10.967319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4b2ed-6d0b-451b-bd3e-0c7bdfa367ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:10.994436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-7fc4b2ed-6d0b-451b-bd3e-0c7bdfa367ad None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:10.995644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7fc4b2ed-6d0b-451b-bd3e-0c7bdfa367ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:11.065706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 638/1275] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:42:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:42:11.330550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d15cc8-2704-4d2e-93bd-fd95ae0aacc8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:11.443398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d15cc8-2704-4d2e-93bd-fd95ae0aacc8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:11.460790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:11.484215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:11.484841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:11.485123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:11.485394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:11.496076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:42:11.501186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:11.502247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:11.503136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 638/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Oct 4 08:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:11.508685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4b2ed-6d0b-451b-bd3e-0c7bdfa367ad None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:11.594687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:11.616629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4b2ed-6d0b-451b-bd3e-0c7bdfa367ad None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:11.653959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:11.654591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:11.654869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:11.655161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:11.655822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] RBAC: Adding query filter params (name=admin) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:11.656161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:11.657518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:11.665705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 639/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Oct 4 08:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:11.671670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:11.779576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51e37d-539a-480d-90be-b6ef249a7026 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:11.820877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:11.821493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:11.821766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:11.822029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:11.832083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:11.833561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:42:11.848067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:11.849167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:42:11.850264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:42:11.850264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:11.850264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:11.850264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 639/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:11.857419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:11.968733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d629c6-c7d1-4c20-acc1-ed8a41b7066a None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:12.004075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:12.004673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:12.004948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:12.005215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:12.005844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:12.006139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:12.007476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:12.019795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 640/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:12.026392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:12.057577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:12.132971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2dd88-0c5b-42da-94bf-f3eeb0062a9b None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:12.167412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:12.168045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:12.168319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:12.168585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:12.176085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:12.177512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:42:12.189213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:12.190268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:12.191651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 640/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:12.196828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:12.200963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:12.301153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c2758-56da-4c2d-a37e-cb9dade13e38 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:12.336190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:12.336783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:12.337053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:12.337319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:12.338035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:12.338324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:12.339724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:12.356473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 641/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Oct 4 08:42:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:12.362976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:12.459004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:12.471070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b4a7f-a1ad-403c-993a-667d9bc381cf None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:12.508218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:12.508810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:12.509085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:12.509349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:12.589027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:12.590459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:12.685934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:12.703339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 641/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Oct 4 08:42:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:42:12.711824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:12.778797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:12.818245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af29eff-983a-4f19-8ea7-d551d5f5dc64 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:12.854838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:12.855449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:12.855730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:12.856035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:12.866528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:12.867989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:42:12.883208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:12.884318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:42:12.886230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:42:12.886230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:42:12.886230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:12.886230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 642/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:12.893359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:13.016694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc643d70-fcd8-40c4-a355-bc9e9f6dc56d None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:13.051874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:13.052483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:13.052753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:13.053020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:13.053719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:13.054007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:13.055361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:13.066480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 642/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Oct 4 08:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:13.073905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:13.177765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf00e-1da2-4ae6-9f81-122db8c2a8b3 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:13.191270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:13.213450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:13.214059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:13.214331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:13.214592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:13.221617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:13.222970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:42:13.234273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:13.235306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: ERROR keystone.server.flask.application Oct 04 08:42:13.236600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 643/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Oct 4 08:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:13.241456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:13.295450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:13.349725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba112049-e445-4c07-9650-14222ac18862 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:13.384305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b7bfa-5b3a-4bf8-8d67-66202daedee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:13.384901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7bfa-5b3a-4bf8-8d67-66202daedee9 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:13.385177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7bfa-5b3a-4bf8-8d67-66202daedee9 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:13.385426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7bfa-5b3a-4bf8-8d67-66202daedee9 None admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:13.386110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b7bfa-5b3a-4bf8-8d67-66202daedee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:13.386408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b7bfa-5b3a-4bf8-8d67-66202daedee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:13.387842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b7bfa-5b3a-4bf8-8d67-66202daedee9 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:13.399688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 643/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Oct 4 08:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:13.406274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:13.511266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d32a8-bfb3-43a2-b1a2-f26473203058 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:13.545811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4406621-2342-4312-b146-74cb1575d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:13.546399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406621-2342-4312-b146-74cb1575d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:13.546675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406621-2342-4312-b146-74cb1575d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:13.546949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4406621-2342-4312-b146-74cb1575d3e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:13.547665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4406621-2342-4312-b146-74cb1575d3e1 None admin] RBAC: Adding query filter params (role.id=3341c8a56a534032a6359202e4ab0ba7, scope.project.id=c88e95e15cef4d41976389c691f4bca4, user.id=9cee970f95754740b2f19e64f4653c7f) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:13.547999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4406621-2342-4312-b146-74cb1575d3e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:13.549411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4406621-2342-4312-b146-74cb1575d3e1 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:13.555335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 644/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Oct 4 08:42:13 2022] GET /identity/v3/role_assignments?user.id=9cee970f95754740b2f19e64f4653c7f&scope.project.id=c88e95e15cef4d41976389c691f4bca4&role.id=3341c8a56a534032a6359202e4ab0ba7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:15.685985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc42d8-9a34-4ce8-b3da-69fe8eef2d82 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:15.686541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc42d8-9a34-4ce8-b3da-69fe8eef2d82 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:15.686541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc42d8-9a34-4ce8-b3da-69fe8eef2d82 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:15.687857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 644/1288] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:15.693267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854ca0-0479-4406-ba54-65a8e85f6d65 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:15.693729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854ca0-0479-4406-ba54-65a8e85f6d65 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:15.693877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854ca0-0479-4406-ba54-65a8e85f6d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:15.714428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-71854ca0-0479-4406-ba54-65a8e85f6d65 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:15.715599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-71854ca0-0479-4406-ba54-65a8e85f6d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:15.747256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:15.790366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 645/1289] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:42:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:42:16.055657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc42d8-9a34-4ce8-b3da-69fe8eef2d82 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:16.161518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc42d8-9a34-4ce8-b3da-69fe8eef2d82 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:16.172047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:16.196848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:16.197458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:16.197734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:16.198005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] PATH_INFO: `/v3/services/placement` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:16.198580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:16.199989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:16.208555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:16.209775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:16.210776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 645/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:42:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:16.216189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71854ca0-0479-4406-ba54-65a8e85f6d65 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:16.325555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71854ca0-0479-4406-ba54-65a8e85f6d65 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:16.360049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a94ab4-1b68-4976-a8eb-d23833546995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:16.360630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a94ab4-1b68-4976-a8eb-d23833546995 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:16.360904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a94ab4-1b68-4976-a8eb-d23833546995 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:16.361170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a94ab4-1b68-4976-a8eb-d23833546995 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:16.361785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a94ab4-1b68-4976-a8eb-d23833546995 None admin] RBAC: Adding query filter params (name=placement) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:16.362080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a94ab4-1b68-4976-a8eb-d23833546995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:16.363419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a94ab4-1b68-4976-a8eb-d23833546995 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:16.368337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 646/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Oct 4 08:42:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:16.377733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:16.488995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090307-b0f9-4596-a144-ca3f72dbb490 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:16.524880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd92cc-dedd-4f45-8a54-f4c61b380531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:16.525461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd92cc-dedd-4f45-8a54-f4c61b380531 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:16.525736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd92cc-dedd-4f45-8a54-f4c61b380531 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:16.526005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd92cc-dedd-4f45-8a54-f4c61b380531 None admin] PATH_INFO: `/v3/services` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:16.526627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd92cc-dedd-4f45-8a54-f4c61b380531 None admin] RBAC: Adding query filter params (type=placement) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:16.526926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd92cc-dedd-4f45-8a54-f4c61b380531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:16.528322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd92cc-dedd-4f45-8a54-f4c61b380531 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:16.532812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 646/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Oct 4 08:42:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:18.623164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a885db2-ba81-40f8-a477-2ae7a8a35d12 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:18.624199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a885db2-ba81-40f8-a477-2ae7a8a35d12 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:18.624199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a885db2-ba81-40f8-a477-2ae7a8a35d12 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:18.626810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 647/1293] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:42:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:18.632899 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e9df9-d512-4f54-b32b-402c73e0cf76 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:18.633357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e9df9-d512-4f54-b32b-402c73e0cf76 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:18.633478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e9df9-d512-4f54-b32b-402c73e0cf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:18.654048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-912e9df9-d512-4f54-b32b-402c73e0cf76 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:18.655306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-912e9df9-d512-4f54-b32b-402c73e0cf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:18.686159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:18.725675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 647/1294] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:42:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:42:18.986387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a885db2-ba81-40f8-a477-2ae7a8a35d12 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:19.025667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:19.103118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a885db2-ba81-40f8-a477-2ae7a8a35d12 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:19.139067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97deda-6a80-4ea8-bf9b-1e669436d7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:19.139656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97deda-6a80-4ea8-bf9b-1e669436d7fd None admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:19.139922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97deda-6a80-4ea8-bf9b-1e669436d7fd None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:19.140224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97deda-6a80-4ea8-bf9b-1e669436d7fd None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:19.140865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97deda-6a80-4ea8-bf9b-1e669436d7fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:19.142199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97deda-6a80-4ea8-bf9b-1e669436d7fd None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:19.159944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 648/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Oct 4 08:42:18 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:42:21.306440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08858b67-d0f9-4bd2-90b8-443bdec8d8ba None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:21.307093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08858b67-d0f9-4bd2-90b8-443bdec8d8ba None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:21.307093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08858b67-d0f9-4bd2-90b8-443bdec8d8ba None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:21.308378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 648/1296] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:21.314594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61bb9-3c3e-4130-9e47-eabae37f762f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:21.315351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61bb9-3c3e-4130-9e47-eabae37f762f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:21.315513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61bb9-3c3e-4130-9e47-eabae37f762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:21.336593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-80b61bb9-3c3e-4130-9e47-eabae37f762f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:21.337811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-80b61bb9-3c3e-4130-9e47-eabae37f762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:21.430179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 649/1297] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:42:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:42:21.692329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08858b67-d0f9-4bd2-90b8-443bdec8d8ba None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:21.797742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08858b67-d0f9-4bd2-90b8-443bdec8d8ba None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:21.832944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:21.833545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:21.833814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:21.834083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:21.835657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:21.836119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:21.845244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:21.846452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:21.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 649/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:42:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:21.853279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b61bb9-3c3e-4130-9e47-eabae37f762f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:21.877855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:21.964142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b61bb9-3c3e-4130-9e47-eabae37f762f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:21.999314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:21.999913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:22.000506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:22.000809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:22.001431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] RBAC: Adding query filter params (name=placement) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:22.001729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:22.003076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:22.008396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 650/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Oct 4 08:42:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:42:22.014529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:22.052928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:22.136401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf76d4-8bc6-4a3b-973b-2e33e82236a8 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:22.171039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e4bf4-d90c-4dd4-ba8d-73fe1b8167c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:22.171633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4bf4-d90c-4dd4-ba8d-73fe1b8167c5 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:22.171926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4bf4-d90c-4dd4-ba8d-73fe1b8167c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:22.172202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4bf4-d90c-4dd4-ba8d-73fe1b8167c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:22.172918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e4bf4-d90c-4dd4-ba8d-73fe1b8167c5 None admin] RBAC: Adding query filter params (interface=public, service_id=bdb08d1fda554071a48c0c543adba36a, region_id=RegionOne) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:22.173223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e4bf4-d90c-4dd4-ba8d-73fe1b8167c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:22.174590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e4bf4-d90c-4dd4-ba8d-73fe1b8167c5 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:22.179652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 650/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Oct 4 08:42:22 2022] GET /identity/v3/endpoints?service_id=bdb08d1fda554071a48c0c543adba36a&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:22.188265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:22.275167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:22.296890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e43a-2e2b-4373-b370-84cdf4498d00 None admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:22.333433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af78ad0-2fe8-4a20-9655-a81b0b00ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:22.334042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78ad0-2fe8-4a20-9655-a81b0b00ad33 None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:22.334290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78ad0-2fe8-4a20-9655-a81b0b00ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:22.334610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af78ad0-2fe8-4a20-9655-a81b0b00ad33 None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:22.335230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af78ad0-2fe8-4a20-9655-a81b0b00ad33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:22.336613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af78ad0-2fe8-4a20-9655-a81b0b00ad33 None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:22.343412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 651/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Oct 4 08:42:22 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:42:24.454623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48dbd3-88dc-4798-95ad-1127b50207c9 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:24.455515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48dbd3-88dc-4798-95ad-1127b50207c9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:24.455515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48dbd3-88dc-4798-95ad-1127b50207c9 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:24.457623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 651/1302] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:42:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:24.466354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50e155-588c-4286-893f-a7a3c4cad9d5 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:24.466779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50e155-588c-4286-893f-a7a3c4cad9d5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:24.466972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50e155-588c-4286-893f-a7a3c4cad9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:24.488925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9b50e155-588c-4286-893f-a7a3c4cad9d5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:24.490098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9b50e155-588c-4286-893f-a7a3c4cad9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:24.560499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 652/1303] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:42:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:42:24.826764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48dbd3-88dc-4798-95ad-1127b50207c9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:24.927371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:24.935702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48dbd3-88dc-4798-95ad-1127b50207c9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:24.976319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:24.977029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:24.977310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:24.977570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] PATH_INFO: `/v3/services/placement` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:24.978203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:24.979613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:42:24.989643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:42:24.990733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application with Lock( Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: ERROR keystone.server.flask.application Oct 04 08:42:24.991844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 652/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Oct 4 08:42:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:42:24.997731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50e155-588c-4286-893f-a7a3c4cad9d5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:25.077985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:25.106007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50e155-588c-4286-893f-a7a3c4cad9d5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:25.149012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cf099-2be9-4426-9b97-79b57a5ccccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:25.149880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cf099-2be9-4426-9b97-79b57a5ccccc None admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:25.150244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cf099-2be9-4426-9b97-79b57a5ccccc None admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:25.150596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cf099-2be9-4426-9b97-79b57a5ccccc None admin] PATH_INFO: `/v3/services` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:25.151540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cf099-2be9-4426-9b97-79b57a5ccccc None admin] RBAC: Adding query filter params (name=placement) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:25.151971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cf099-2be9-4426-9b97-79b57a5ccccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:25.154060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cf099-2be9-4426-9b97-79b57a5ccccc None admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:25.162969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 653/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Oct 4 08:42:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:25.169169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:25.274579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2dcb2-63ed-4fbe-89d9-da1ee945dd46 None admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:25.309634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf88db9-08a6-4044-9b69-42cee477242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:25.310249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf88db9-08a6-4044-9b69-42cee477242e None admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:25.310534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf88db9-08a6-4044-9b69-42cee477242e None admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:25.310805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf88db9-08a6-4044-9b69-42cee477242e None admin] PATH_INFO: `/v3/endpoints` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:25.311525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf88db9-08a6-4044-9b69-42cee477242e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:25.313027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf88db9-08a6-4044-9b69-42cee477242e None admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:25.335985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 653/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:42:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:42:35.157397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=30 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:35.330689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:29.389766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f525-91fc-4c75-987d-9507503668ac None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:29.390253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f525-91fc-4c75-987d-9507503668ac None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:29.390486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f525-91fc-4c75-987d-9507503668ac None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:29.397988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f7b8-415d-410a-b91b-9dc45c359c73 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:29.397988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f7b8-415d-410a-b91b-9dc45c359c73 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:29.398419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f7b8-415d-410a-b91b-9dc45c359c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:29.400460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 654/1307] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:29.518545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-db63f7b8-415d-410a-b91b-9dc45c359c73 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:29.520789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-db63f7b8-415d-410a-b91b-9dc45c359c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:29.716323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 654/1308] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:30.020308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c863-e3b9-4194-9dea-ca2fbb30095c None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:30.020838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c863-e3b9-4194-9dea-ca2fbb30095c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:30.021281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c863-e3b9-4194-9dea-ca2fbb30095c None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:30.023123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 655/1309] 10.222.0.44 () {54 vars in 964 bytes} [Tue Oct 4 08:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:30.029820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d64863-cdf3-4527-ad4d-b46fa13ff5ce None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:30.030117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d64863-cdf3-4527-ad4d-b46fa13ff5ce None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:30.030402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d64863-cdf3-4527-ad4d-b46fa13ff5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:30.129417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-45d64863-cdf3-4527-ad4d-b46fa13ff5ce None None] MFA Rules not processed for user `349ee49c297749468e09c4aa3dc9c218`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:30.130787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-45d64863-cdf3-4527-ad4d-b46fa13ff5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:30.156559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:30.226763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 655/1310] 10.222.0.44 () {58 vars in 1061 bytes} [Tue Oct 4 08:43:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:30.236227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c863-e3b9-4194-9dea-ca2fbb30095c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:30.238274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-dde6c863-e3b9-4194-9dea-ca2fbb30095c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:30.367014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c863-e3b9-4194-9dea-ca2fbb30095c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:30.451292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575662ff-a3ac-422a-ad69-4a2639738261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:30.451924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575662ff-a3ac-422a-ad69-4a2639738261 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:30.452130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575662ff-a3ac-422a-ad69-4a2639738261 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:30.452580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575662ff-a3ac-422a-ad69-4a2639738261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:30.454340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-575662ff-a3ac-422a-ad69-4a2639738261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:30.515920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:30.575991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575662ff-a3ac-422a-ad69-4a2639738261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:30.577683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575662ff-a3ac-422a-ad69-4a2639738261 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:30.598830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 656/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:33.014030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fdaeb-ad20-4132-a691-acb85b830eb5 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:33.014462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fdaeb-ad20-4132-a691-acb85b830eb5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:33.014615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fdaeb-ad20-4132-a691-acb85b830eb5 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:33.016304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 656/1312] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:33.021905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd842-b67f-44d6-b72a-ed1478ae9401 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:33.022306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd842-b67f-44d6-b72a-ed1478ae9401 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:33.022598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd842-b67f-44d6-b72a-ed1478ae9401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:33.107744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d61dd842-b67f-44d6-b72a-ed1478ae9401 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:33.108989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d61dd842-b67f-44d6-b72a-ed1478ae9401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:33.212548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 657/1313] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:33.497370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fdaeb-ad20-4132-a691-acb85b830eb5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:33.499868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3f0fdaeb-ad20-4132-a691-acb85b830eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:33.613865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fdaeb-ad20-4132-a691-acb85b830eb5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:33.688651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:33.691042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4cc23-0813-4a0c-917e-930554281c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:33.691667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4cc23-0813-4a0c-917e-930554281c1e service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:33.691940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4cc23-0813-4a0c-917e-930554281c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:33.692231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4cc23-0813-4a0c-917e-930554281c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:33.694424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-53f4cc23-0813-4a0c-917e-930554281c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:33.817607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4cc23-0813-4a0c-917e-930554281c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:33.819029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4cc23-0813-4a0c-917e-930554281c1e service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:33.836850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 657/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:36.278767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0a38-e843-4dad-aff8-4dee5905262b None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:36.279334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0a38-e843-4dad-aff8-4dee5905262b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:36.279334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0a38-e843-4dad-aff8-4dee5905262b None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:36.280704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 658/1315] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:36.287096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3f59-076f-4f1e-8589-7b4d61693265 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:36.287862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3f59-076f-4f1e-8589-7b4d61693265 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:36.288065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3f59-076f-4f1e-8589-7b4d61693265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:36.356235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-23da3f59-076f-4f1e-8589-7b4d61693265 None None] MFA Rules not processed for user `e2fd43b4750842c794560ebef09e5f94`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:36.357443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-23da3f59-076f-4f1e-8589-7b4d61693265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:36.410287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:36.455481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 658/1316] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:36.748133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444c0a-951a-457f-89fb-5bf4eb2ec95a None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:36.748389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444c0a-951a-457f-89fb-5bf4eb2ec95a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:36.748688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444c0a-951a-457f-89fb-5bf4eb2ec95a None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:36.750114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 659/1317] 10.222.0.44 () {54 vars in 964 bytes} [Tue Oct 4 08:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:36.757038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a5aa6-b43d-4136-8c0c-eed69eadbbf4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:36.757353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a5aa6-b43d-4136-8c0c-eed69eadbbf4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:36.757604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a5aa6-b43d-4136-8c0c-eed69eadbbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:36.841616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8b6a5aa6-b43d-4136-8c0c-eed69eadbbf4 None None] MFA Rules not processed for user `349ee49c297749468e09c4aa3dc9c218`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:36.842813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8b6a5aa6-b43d-4136-8c0c-eed69eadbbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:36.921506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 659/1318] 10.222.0.44 () {58 vars in 1061 bytes} [Tue Oct 4 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:36.930846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444c0a-951a-457f-89fb-5bf4eb2ec95a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:36.932398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-cc444c0a-951a-457f-89fb-5bf4eb2ec95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:36.942660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:37.040109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444c0a-951a-457f-89fb-5bf4eb2ec95a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:37.119561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f41dc-a6d7-4fcf-87bc-70523cf31b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:37.120200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f41dc-a6d7-4fcf-87bc-70523cf31b9c service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:37.120500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f41dc-a6d7-4fcf-87bc-70523cf31b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:37.120748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f41dc-a6d7-4fcf-87bc-70523cf31b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:37.122357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e88f41dc-a6d7-4fcf-87bc-70523cf31b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:37.187716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:37.236926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f41dc-a6d7-4fcf-87bc-70523cf31b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:37.238362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f41dc-a6d7-4fcf-87bc-70523cf31b9c service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:37.257255 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 660/1319] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:40.332321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f24bd7-c343-4d02-bb7e-452f49c172e5 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:40.332955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f24bd7-c343-4d02-bb7e-452f49c172e5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:40.332955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f24bd7-c343-4d02-bb7e-452f49c172e5 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:40.335070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 660/1320] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:40.341401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbc7cc-7c11-4a29-bf86-8e45d1824f20 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:40.341777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbc7cc-7c11-4a29-bf86-8e45d1824f20 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:40.341949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbc7cc-7c11-4a29-bf86-8e45d1824f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:40.418667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-85fbc7cc-7c11-4a29-bf86-8e45d1824f20 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:40.420485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-85fbc7cc-7c11-4a29-bf86-8e45d1824f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:40.525363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 661/1321] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:40.819560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f24bd7-c343-4d02-bb7e-452f49c172e5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:40.821641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-31f24bd7-c343-4d02-bb7e-452f49c172e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:40.871687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:40.928487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f24bd7-c343-4d02-bb7e-452f49c172e5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:41.006909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b5b40-a479-4ccf-9f1a-468cabb0062e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:41.007527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b5b40-a479-4ccf-9f1a-468cabb0062e service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:41.007806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b5b40-a479-4ccf-9f1a-468cabb0062e service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:41.008142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b5b40-a479-4ccf-9f1a-468cabb0062e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:41.009781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-300b5b40-a479-4ccf-9f1a-468cabb0062e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:41.116981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:41.131400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b5b40-a479-4ccf-9f1a-468cabb0062e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:41.132861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b5b40-a479-4ccf-9f1a-468cabb0062e service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:41.152594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 661/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:43.966057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3a34f-2141-429e-99b7-a0d854386338 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:43.966608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3a34f-2141-429e-99b7-a0d854386338 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:43.966608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3a34f-2141-429e-99b7-a0d854386338 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:43.968186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 662/1323] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:43.973712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39057740-8591-423b-ab0d-30e93e1d3401 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:43.974138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39057740-8591-423b-ab0d-30e93e1d3401 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:43.974225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39057740-8591-423b-ab0d-30e93e1d3401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:44.051577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-39057740-8591-423b-ab0d-30e93e1d3401 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:44.052849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-39057740-8591-423b-ab0d-30e93e1d3401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:44.168930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 662/1324] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:44.448353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3a34f-2141-429e-99b7-a0d854386338 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:44.450049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9aa3a34f-2141-429e-99b7-a0d854386338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:44.470819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:44.559981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3a34f-2141-429e-99b7-a0d854386338 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:44.638199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3756887-c672-4e27-ac95-7962951230b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:44.638807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756887-c672-4e27-ac95-7962951230b4 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:44.639060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756887-c672-4e27-ac95-7962951230b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:44.639360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3756887-c672-4e27-ac95-7962951230b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:44.641014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b3756887-c672-4e27-ac95-7962951230b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:44.714976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:44.756890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3756887-c672-4e27-ac95-7962951230b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:44.758421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3756887-c672-4e27-ac95-7962951230b4 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:44.776909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 663/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:46.982665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8210-ff45-4194-9748-0c31eeea9f46 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:46.983147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8210-ff45-4194-9748-0c31eeea9f46 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:46.983267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8210-ff45-4194-9748-0c31eeea9f46 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:46.984582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 663/1326] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:46.990170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345afe25-f7c2-42c4-8d8d-030b0d58dfd8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:46.990578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345afe25-f7c2-42c4-8d8d-030b0d58dfd8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:46.990843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345afe25-f7c2-42c4-8d8d-030b0d58dfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:47.062621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-345afe25-f7c2-42c4-8d8d-030b0d58dfd8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:47.066924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-345afe25-f7c2-42c4-8d8d-030b0d58dfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:47.177651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 664/1327] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:47.461135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8210-ff45-4194-9748-0c31eeea9f46 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:47.462800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-bf4c8210-ff45-4194-9748-0c31eeea9f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:47.527367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:47.573626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8210-ff45-4194-9748-0c31eeea9f46 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:47.651062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ce001-d1d0-48a0-8893-d0d2aa9d9d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:47.651670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ce001-d1d0-48a0-8893-d0d2aa9d9d52 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:47.651950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ce001-d1d0-48a0-8893-d0d2aa9d9d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:47.652239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ce001-d1d0-48a0-8893-d0d2aa9d9d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:47.653888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-652ce001-d1d0-48a0-8893-d0d2aa9d9d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:47.765668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:47.768008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ce001-d1d0-48a0-8893-d0d2aa9d9d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:47.769715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ce001-d1d0-48a0-8893-d0d2aa9d9d52 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:47.788184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 664/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:49.973799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eda20-939f-4951-96ac-7d5349c8e320 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:49.974254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eda20-939f-4951-96ac-7d5349c8e320 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:49.974373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eda20-939f-4951-96ac-7d5349c8e320 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:49.976632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 665/1329] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:49.982482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53ec30-e8d9-448d-a6a6-e973a99c10a0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:49.982881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53ec30-e8d9-448d-a6a6-e973a99c10a0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:49.983022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53ec30-e8d9-448d-a6a6-e973a99c10a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:50.052375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5d53ec30-e8d9-448d-a6a6-e973a99c10a0 None None] MFA Rules not processed for user `e2fd43b4750842c794560ebef09e5f94`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:50.053568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5d53ec30-e8d9-448d-a6a6-e973a99c10a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:50.152232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 665/1330] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:50.436454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eda20-939f-4951-96ac-7d5349c8e320 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:50.438429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fa1eda20-939f-4951-96ac-7d5349c8e320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:50.473591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:50.547037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eda20-939f-4951-96ac-7d5349c8e320 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:50.624322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf65866-9482-4a80-835a-70f6d7e3883a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:50.625240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf65866-9482-4a80-835a-70f6d7e3883a service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:50.625516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf65866-9482-4a80-835a-70f6d7e3883a service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:50.625784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf65866-9482-4a80-835a-70f6d7e3883a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:50.627383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bcf65866-9482-4a80-835a-70f6d7e3883a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:50.704865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:50.738748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf65866-9482-4a80-835a-70f6d7e3883a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:50.740125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf65866-9482-4a80-835a-70f6d7e3883a service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:50.759921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 666/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:53.149461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67f6a-d112-4164-9850-ed0bd8f12332 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:53.149960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67f6a-d112-4164-9850-ed0bd8f12332 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:53.149960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67f6a-d112-4164-9850-ed0bd8f12332 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:53.151563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 666/1332] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:53.158965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a65bc-6dcf-48c7-ac0f-b08846e3e139 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:53.159406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a65bc-6dcf-48c7-ac0f-b08846e3e139 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:53.159536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a65bc-6dcf-48c7-ac0f-b08846e3e139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:53.236644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-942a65bc-6dcf-48c7-ac0f-b08846e3e139 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:53.237776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-942a65bc-6dcf-48c7-ac0f-b08846e3e139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:53.347091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 667/1333] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:53.633975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67f6a-d112-4164-9850-ed0bd8f12332 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:53.636500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-60b67f6a-d112-4164-9850-ed0bd8f12332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:53.717316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:53.752446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67f6a-d112-4164-9850-ed0bd8f12332 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:53.827974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cab45-464f-4e6f-a861-adf9e5e4ba79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:53.828585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cab45-464f-4e6f-a861-adf9e5e4ba79 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:53.828859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cab45-464f-4e6f-a861-adf9e5e4ba79 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:53.829125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cab45-464f-4e6f-a861-adf9e5e4ba79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:53.830744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4f6cab45-464f-4e6f-a861-adf9e5e4ba79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:53.959019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cab45-464f-4e6f-a861-adf9e5e4ba79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:53.960407 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cab45-464f-4e6f-a861-adf9e5e4ba79 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:53.971077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:53.980152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 667/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:56.177794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff803c4-8654-49bb-8000-5bb3e54229b5 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:56.178373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff803c4-8654-49bb-8000-5bb3e54229b5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:56.178373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff803c4-8654-49bb-8000-5bb3e54229b5 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:56.179689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 668/1335] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:56.186269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1a3f-c71a-47a4-9f31-a41beb2e700f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:56.186994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1a3f-c71a-47a4-9f31-a41beb2e700f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:56.187187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc1a3f-c71a-47a4-9f31-a41beb2e700f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:56.263762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c0cc1a3f-c71a-47a4-9f31-a41beb2e700f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:56.264900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c0cc1a3f-c71a-47a4-9f31-a41beb2e700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:56.364624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 668/1336] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:56.648568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff803c4-8654-49bb-8000-5bb3e54229b5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:56.650198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5ff803c4-8654-49bb-8000-5bb3e54229b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:56.692479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:56.759341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff803c4-8654-49bb-8000-5bb3e54229b5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:56.844310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334da5f-293e-459b-b8f5-143cdc8f5790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:56.845156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334da5f-293e-459b-b8f5-143cdc8f5790 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:56.845515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334da5f-293e-459b-b8f5-143cdc8f5790 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:56.845874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334da5f-293e-459b-b8f5-143cdc8f5790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:56.848510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d334da5f-293e-459b-b8f5-143cdc8f5790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:56.947860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:56.976308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334da5f-293e-459b-b8f5-143cdc8f5790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:56.977804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334da5f-293e-459b-b8f5-143cdc8f5790 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:56.996850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 669/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:59.651460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87361ff9-5379-4630-b61b-ca1bea0fc9b0 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:59.652421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87361ff9-5379-4630-b61b-ca1bea0fc9b0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:59.652421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87361ff9-5379-4630-b61b-ca1bea0fc9b0 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:59.654543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 669/1338] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:43:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:59.660718 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd480b-350b-419e-9411-aa0a7728b3d9 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:59.661166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd480b-350b-419e-9411-aa0a7728b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:59.661274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd480b-350b-419e-9411-aa0a7728b3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:59.732019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-84dd480b-350b-419e-9411-aa0a7728b3d9 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:59.733258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-84dd480b-350b-419e-9411-aa0a7728b3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:59.835104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 670/1339] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:43:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:00.116958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87361ff9-5379-4630-b61b-ca1bea0fc9b0 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:00.118921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-87361ff9-5379-4630-b61b-ca1bea0fc9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:00.198064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:00.235225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87361ff9-5379-4630-b61b-ca1bea0fc9b0 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:00.310381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cac325-722b-4786-8e50-f703af9dc024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:00.310981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cac325-722b-4786-8e50-f703af9dc024 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:00.311260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cac325-722b-4786-8e50-f703af9dc024 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:00.311524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cac325-722b-4786-8e50-f703af9dc024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:00.313169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-97cac325-722b-4786-8e50-f703af9dc024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:00.441755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cac325-722b-4786-8e50-f703af9dc024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:00.443195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cac325-722b-4786-8e50-f703af9dc024 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:00.462086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 670/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:04.615469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ff10f-4ab4-4039-bd57-1482e69d32c1 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:04.616046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ff10f-4ab4-4039-bd57-1482e69d32c1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:04.616046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ff10f-4ab4-4039-bd57-1482e69d32c1 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:04.617333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 671/1341] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:04.622897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b948e8-894f-4c64-8ba3-3ae600526b46 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:04.623335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b948e8-894f-4c64-8ba3-3ae600526b46 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:04.623450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b948e8-894f-4c64-8ba3-3ae600526b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:04.629466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:04.689905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-36b948e8-894f-4c64-8ba3-3ae600526b46 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:04.691037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-36b948e8-894f-4c64-8ba3-3ae600526b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:04.788791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 671/1342] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:05.063651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ff10f-4ab4-4039-bd57-1482e69d32c1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:05.065220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-941ff10f-4ab4-4039-bd57-1482e69d32c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:05.115175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:05.171848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ff10f-4ab4-4039-bd57-1482e69d32c1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:05.248065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845f14e-fbca-4693-8ed2-95d9c391e47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:05.248601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845f14e-fbca-4693-8ed2-95d9c391e47a service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:05.248877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845f14e-fbca-4693-8ed2-95d9c391e47a service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:05.249139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845f14e-fbca-4693-8ed2-95d9c391e47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:05.250731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0845f14e-fbca-4693-8ed2-95d9c391e47a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:05.345347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:05.364003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845f14e-fbca-4693-8ed2-95d9c391e47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:05.365426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845f14e-fbca-4693-8ed2-95d9c391e47a service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:05.384821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 672/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:08.161933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59d4b5-d58b-45de-bcc3-d635393fbc2d None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:08.162467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59d4b5-d58b-45de-bcc3-d635393fbc2d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:08.162559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59d4b5-d58b-45de-bcc3-d635393fbc2d None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:08.164010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 672/1344] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:08.169443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ae7b3-7165-4540-bc07-2cf0430739e8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:08.169879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ae7b3-7165-4540-bc07-2cf0430739e8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:08.169991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ae7b3-7165-4540-bc07-2cf0430739e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:08.236976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-de5ae7b3-7165-4540-bc07-2cf0430739e8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:08.238091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-de5ae7b3-7165-4540-bc07-2cf0430739e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:08.339110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 673/1345] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:08.622240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59d4b5-d58b-45de-bcc3-d635393fbc2d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:08.624237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9f59d4b5-d58b-45de-bcc3-d635393fbc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:08.721984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:08.740130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59d4b5-d58b-45de-bcc3-d635393fbc2d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:08.818329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbbbb2-8c64-44e7-a619-24826560631f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:08.818962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbbb2-8c64-44e7-a619-24826560631f service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:08.819240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbbb2-8c64-44e7-a619-24826560631f service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:08.819512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbbb2-8c64-44e7-a619-24826560631f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:08.821219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-86dbbbb2-8c64-44e7-a619-24826560631f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:08.934214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbbbb2-8c64-44e7-a619-24826560631f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:08.935665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbbbb2-8c64-44e7-a619-24826560631f service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:08.955254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 673/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:13.033989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a610-2b7c-48c6-a4e9-59a91901169a None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:13.034452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a610-2b7c-48c6-a4e9-59a91901169a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:13.034627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a610-2b7c-48c6-a4e9-59a91901169a None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:13.036026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 674/1347] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:13.041785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d53d1-ab98-4b52-baf9-df7d7781045c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:13.042206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d53d1-ab98-4b52-baf9-df7d7781045c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:13.042552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d53d1-ab98-4b52-baf9-df7d7781045c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:13.060542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:13.118591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-db0d53d1-ab98-4b52-baf9-df7d7781045c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:13.119829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-db0d53d1-ab98-4b52-baf9-df7d7781045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:13.225166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 674/1348] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:44:13.520912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a610-2b7c-48c6-a4e9-59a91901169a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:13.522547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6446a610-2b7c-48c6-a4e9-59a91901169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:13.586684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:13.633396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a610-2b7c-48c6-a4e9-59a91901169a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:13.708676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771c53b-06d0-4ceb-b327-23b395fb50cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:13.709277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771c53b-06d0-4ceb-b327-23b395fb50cf service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:13.709548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771c53b-06d0-4ceb-b327-23b395fb50cf service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:13.709815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771c53b-06d0-4ceb-b327-23b395fb50cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:13.711395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d771c53b-06d0-4ceb-b327-23b395fb50cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:13.824513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:13.827932 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771c53b-06d0-4ceb-b327-23b395fb50cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:13.829375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771c53b-06d0-4ceb-b327-23b395fb50cf service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:13.848066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 675/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:18.697071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b2c2c-d787-425f-ab19-6885294667b6 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:18.698014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b2c2c-d787-425f-ab19-6885294667b6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:18.698515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b2c2c-d787-425f-ab19-6885294667b6 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:18.700425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 675/1350] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:18.707033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc400af-2c60-44ee-990e-cfc040462225 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:18.707033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc400af-2c60-44ee-990e-cfc040462225 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:18.707033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc400af-2c60-44ee-990e-cfc040462225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:18.793980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-dcc400af-2c60-44ee-990e-cfc040462225 None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:18.795389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-dcc400af-2c60-44ee-990e-cfc040462225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:18.991559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 676/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:44:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:23.095473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77f2a5-2c52-4186-80ab-512b4f548ec0 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:23.096022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77f2a5-2c52-4186-80ab-512b4f548ec0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:23.096309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77f2a5-2c52-4186-80ab-512b4f548ec0 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:23.097740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 676/1352] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:23.103036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a923a5-fcae-4749-ad4f-974af1c4973a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:23.103446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a923a5-fcae-4749-ad4f-974af1c4973a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:23.103637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a923a5-fcae-4749-ad4f-974af1c4973a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:23.168063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:23.194342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-07a923a5-fcae-4749-ad4f-974af1c4973a None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:23.195526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-07a923a5-fcae-4749-ad4f-974af1c4973a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:23.223692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=59 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:23.309742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 677/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:24.727685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f9a2-c92f-4436-bb56-91c8c3c305ed None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:24.728312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f9a2-c92f-4436-bb56-91c8c3c305ed None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:24.728423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f9a2-c92f-4436-bb56-91c8c3c305ed None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:24.729835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 677/1354] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:24.735075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ba70e-169a-491a-b6c9-4f9a8436d933 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:24.735489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ba70e-169a-491a-b6c9-4f9a8436d933 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:24.735761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ba70e-169a-491a-b6c9-4f9a8436d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:24.815695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f06ba70e-169a-491a-b6c9-4f9a8436d933 None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:24.816948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f06ba70e-169a-491a-b6c9-4f9a8436d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:24.861129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:24.944756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 678/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:28.300535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9c771-eb2a-4101-b6ff-45141d2baf53 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:28.301372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9c771-eb2a-4101-b6ff-45141d2baf53 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:28.301858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9c771-eb2a-4101-b6ff-45141d2baf53 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:28.307354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 678/1356] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:28.309378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087579-514a-4da6-a277-5283895e8149 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:28.310178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087579-514a-4da6-a277-5283895e8149 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:28.310616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087579-514a-4da6-a277-5283895e8149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:28.521822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-77087579-514a-4da6-a277-5283895e8149 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:28.523140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-77087579-514a-4da6-a277-5283895e8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:28.628778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 679/1357] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:28.920797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fff841-f3be-439d-8315-d0b99f190a60 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:28.921348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fff841-f3be-439d-8315-d0b99f190a60 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:28.921833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fff841-f3be-439d-8315-d0b99f190a60 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:28.923723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 679/1358] 10.222.0.44 () {54 vars in 961 bytes} [Tue Oct 4 08:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:28.932608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8637-a500-4209-b2f4-9c1a9003ff04 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:28.933129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8637-a500-4209-b2f4-9c1a9003ff04 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:28.933605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8637-a500-4209-b2f4-9c1a9003ff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:28.955863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:29.025279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-767e8637-a500-4209-b2f4-9c1a9003ff04 None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:29.026699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-767e8637-a500-4209-b2f4-9c1a9003ff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:29.149017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 680/1359] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Oct 4 08:44:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:29.157117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fff841-f3be-439d-8315-d0b99f190a60 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:29.158694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b1fff841-f3be-439d-8315-d0b99f190a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:29.318240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fff841-f3be-439d-8315-d0b99f190a60 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:29.391824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:29.433796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80649a93-a5fa-4466-b027-28bf56075594 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:29.434325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80649a93-a5fa-4466-b027-28bf56075594 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:29.434599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80649a93-a5fa-4466-b027-28bf56075594 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:29.434865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80649a93-a5fa-4466-b027-28bf56075594 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:29.436568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-80649a93-a5fa-4466-b027-28bf56075594 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:29.540577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80649a93-a5fa-4466-b027-28bf56075594 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:29.542176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80649a93-a5fa-4466-b027-28bf56075594 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:29.561217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 680/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:31.921360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1581d55-7530-442e-a590-f1301dd02646 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:31.922058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1581d55-7530-442e-a590-f1301dd02646 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:31.922058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1581d55-7530-442e-a590-f1301dd02646 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:31.929438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249b27-c6eb-4e34-9b8a-06ac452cb500 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:31.929438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249b27-c6eb-4e34-9b8a-06ac452cb500 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:31.931501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249b27-c6eb-4e34-9b8a-06ac452cb500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:31.934441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 681/1361] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:32.001871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-30249b27-c6eb-4e34-9b8a-06ac452cb500 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:32.001871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-30249b27-c6eb-4e34-9b8a-06ac452cb500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:32.047292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:32.116047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 681/1362] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:32.397994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1581d55-7530-442e-a590-f1301dd02646 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:32.399847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a1581d55-7530-442e-a590-f1301dd02646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:32.482352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:32.603225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1581d55-7530-442e-a590-f1301dd02646 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:32.725762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43c0d4-447d-4438-8e59-0f267d4200e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:32.726384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c0d4-447d-4438-8e59-0f267d4200e8 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:32.726659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c0d4-447d-4438-8e59-0f267d4200e8 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:32.726927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c0d4-447d-4438-8e59-0f267d4200e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:32.728611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1e43c0d4-447d-4438-8e59-0f267d4200e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:32.752556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:32.822491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43c0d4-447d-4438-8e59-0f267d4200e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:32.824139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43c0d4-447d-4438-8e59-0f267d4200e8 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:32.842653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 682/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:35.315792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23043375-26d5-4d94-9c19-3c7acbf084d1 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:35.316451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23043375-26d5-4d94-9c19-3c7acbf084d1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:35.316451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23043375-26d5-4d94-9c19-3c7acbf084d1 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:35.317807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 682/1364] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:35.323724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a59328-1436-4b1e-ade7-0a9d36ab4f3c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:35.324243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a59328-1436-4b1e-ade7-0a9d36ab4f3c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:35.324347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a59328-1436-4b1e-ade7-0a9d36ab4f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:35.397176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-50a59328-1436-4b1e-ade7-0a9d36ab4f3c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:35.398430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-50a59328-1436-4b1e-ade7-0a9d36ab4f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:35.503304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:35.511949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 683/1365] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:35.796208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fa33-fc5f-40d5-ae0f-9769067ce67a None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:35.796474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fa33-fc5f-40d5-ae0f-9769067ce67a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:35.796937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fa33-fc5f-40d5-ae0f-9769067ce67a None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:35.798512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 683/1366] 10.222.0.44 () {54 vars in 961 bytes} [Tue Oct 4 08:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:35.803703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc474-5b3a-463c-95ac-a605705b1efe None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:35.804100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc474-5b3a-463c-95ac-a605705b1efe None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:35.804444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc474-5b3a-463c-95ac-a605705b1efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:35.887777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0a7bc474-5b3a-463c-95ac-a605705b1efe None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:35.889127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0a7bc474-5b3a-463c-95ac-a605705b1efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:36.019784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 684/1367] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Oct 4 08:44:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:36.027921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fa33-fc5f-40d5-ae0f-9769067ce67a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:36.029488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f424fa33-fc5f-40d5-ae0f-9769067ce67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:36.184555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fa33-fc5f-40d5-ae0f-9769067ce67a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:36.205360 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:36.307980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31246bd-6277-4511-a910-34527998bdd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:36.308597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31246bd-6277-4511-a910-34527998bdd8 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:36.308880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31246bd-6277-4511-a910-34527998bdd8 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:36.309156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31246bd-6277-4511-a910-34527998bdd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:36.310805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c31246bd-6277-4511-a910-34527998bdd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:36.413427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31246bd-6277-4511-a910-34527998bdd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:36.415279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31246bd-6277-4511-a910-34527998bdd8 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:36.435726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 684/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:44:36.505894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa41b6-6c59-4e76-b729-3f2e5ec4e217 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:36.506445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa41b6-6c59-4e76-b729-3f2e5ec4e217 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:36.507218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa41b6-6c59-4e76-b729-3f2e5ec4e217 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:36.509071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 685/1369] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:36.514762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bcaa8-3d4a-4c40-ac05-373d358f6edc None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:36.515083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bcaa8-3d4a-4c40-ac05-373d358f6edc None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:36.515328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bcaa8-3d4a-4c40-ac05-373d358f6edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:36.555378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:36.606953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9d6bcaa8-3d4a-4c40-ac05-373d358f6edc None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:36.608414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9d6bcaa8-3d4a-4c40-ac05-373d358f6edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:36.736028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 685/1370] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:37.062019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d6621-d531-41e0-a0a5-3089d120ff7a None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:37.062265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d6621-d531-41e0-a0a5-3089d120ff7a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:37.062566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d6621-d531-41e0-a0a5-3089d120ff7a None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:37.064138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 686/1371] 10.222.0.44 () {54 vars in 963 bytes} [Tue Oct 4 08:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:37.069093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b485-72c0-40eb-b4bd-2536fbbc0601 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:37.069380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b485-72c0-40eb-b4bd-2536fbbc0601 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:37.069641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b485-72c0-40eb-b4bd-2536fbbc0601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:37.149476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8bb3b485-72c0-40eb-b4bd-2536fbbc0601 None None] MFA Rules not processed for user `d877daa4c2c14d0c8d281aeebab27ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:37.150759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8bb3b485-72c0-40eb-b4bd-2536fbbc0601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:37.201937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:37.241577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 686/1372] 10.222.0.44 () {58 vars in 1060 bytes} [Tue Oct 4 08:44:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:37.245060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d6621-d531-41e0-a0a5-3089d120ff7a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:37.246725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-934d6621-d531-41e0-a0a5-3089d120ff7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:37.301253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:37.359970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d6621-d531-41e0-a0a5-3089d120ff7a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:37.451860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64aebb-155e-4d35-a7fd-3025f9fddb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:37.452691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64aebb-155e-4d35-a7fd-3025f9fddb25 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:37.453067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64aebb-155e-4d35-a7fd-3025f9fddb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:37.453446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64aebb-155e-4d35-a7fd-3025f9fddb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:37.455211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3e64aebb-155e-4d35-a7fd-3025f9fddb25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:37.598993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:37.619602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64aebb-155e-4d35-a7fd-3025f9fddb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:37.621418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64aebb-155e-4d35-a7fd-3025f9fddb25 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:37.653146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 687/1373] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:38.705941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c4ba-522d-472f-988b-2f4b2f22fecd None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:38.706509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c4ba-522d-472f-988b-2f4b2f22fecd None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:38.706509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c4ba-522d-472f-988b-2f4b2f22fecd None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:38.707836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 687/1374] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:38.713620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d19c-4695-4fa9-a088-060400151b99 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:38.714042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d19c-4695-4fa9-a088-060400151b99 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:38.714334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d19c-4695-4fa9-a088-060400151b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:38.784938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9e51d19c-4695-4fa9-a088-060400151b99 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:38.786286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9e51d19c-4695-4fa9-a088-060400151b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:38.890627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 688/1375] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:39.180941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c4ba-522d-472f-988b-2f4b2f22fecd None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:39.182508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7fd0c4ba-522d-472f-988b-2f4b2f22fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:39.338712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c4ba-522d-472f-988b-2f4b2f22fecd None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:39.393466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:39.456434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbfd77-b508-4fb3-9fcc-f6e659d12fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:39.457078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbfd77-b508-4fb3-9fcc-f6e659d12fcb service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:39.457349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbfd77-b508-4fb3-9fcc-f6e659d12fcb service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:39.457617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbfd77-b508-4fb3-9fcc-f6e659d12fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:39.459246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-82dbfd77-b508-4fb3-9fcc-f6e659d12fcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:39.552281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbfd77-b508-4fb3-9fcc-f6e659d12fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:39.553957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbfd77-b508-4fb3-9fcc-f6e659d12fcb service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:39.573112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 688/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:41.780984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386634-c877-4ec5-b865-66dfb4ba0f43 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:41.781539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386634-c877-4ec5-b865-66dfb4ba0f43 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:41.781539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386634-c877-4ec5-b865-66dfb4ba0f43 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:41.783068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 689/1377] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:41.788608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b51d6f-6742-457f-9b9a-18a36cb11a98 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:41.789041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b51d6f-6742-457f-9b9a-18a36cb11a98 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:41.789137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b51d6f-6742-457f-9b9a-18a36cb11a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:41.859401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e1b51d6f-6742-457f-9b9a-18a36cb11a98 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:41.860720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e1b51d6f-6742-457f-9b9a-18a36cb11a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:41.870324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:41.974101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 689/1378] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:42.263337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b386634-c877-4ec5-b865-66dfb4ba0f43 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:42.264895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4b386634-c877-4ec5-b865-66dfb4ba0f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:42.333386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:42.425734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b386634-c877-4ec5-b865-66dfb4ba0f43 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:42.543170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f50ce-2028-40af-acf5-0c2464d46257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:42.544091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f50ce-2028-40af-acf5-0c2464d46257 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:42.544370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f50ce-2028-40af-acf5-0c2464d46257 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:42.544643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f50ce-2028-40af-acf5-0c2464d46257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:42.546214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-055f50ce-2028-40af-acf5-0c2464d46257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:42.565564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:42.650275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f50ce-2028-40af-acf5-0c2464d46257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:42.651924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f50ce-2028-40af-acf5-0c2464d46257 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:42.673668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 690/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:44.872026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8325548-b116-4271-a1f7-b1843009b315 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:44.872540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8325548-b116-4271-a1f7-b1843009b315 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:44.872540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8325548-b116-4271-a1f7-b1843009b315 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:44.877936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 690/1380] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:44.879340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639bd1-9dec-4cda-bf52-1b2467dc13b6 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:44.879762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639bd1-9dec-4cda-bf52-1b2467dc13b6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:44.880297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639bd1-9dec-4cda-bf52-1b2467dc13b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:44.962898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b6639bd1-9dec-4cda-bf52-1b2467dc13b6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:44.964474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b6639bd1-9dec-4cda-bf52-1b2467dc13b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:45.080749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:45.087830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 691/1381] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:45.370703 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8325548-b116-4271-a1f7-b1843009b315 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:45.372521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c8325548-b116-4271-a1f7-b1843009b315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:45.520381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:45.532319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8325548-b116-4271-a1f7-b1843009b315 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:45.654412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82648c70-c4fc-4ee3-9bf9-124fd35281a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:45.654962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648c70-c4fc-4ee3-9bf9-124fd35281a3 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:45.655253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648c70-c4fc-4ee3-9bf9-124fd35281a3 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:45.655537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648c70-c4fc-4ee3-9bf9-124fd35281a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:45.657265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-82648c70-c4fc-4ee3-9bf9-124fd35281a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:45.756624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82648c70-c4fc-4ee3-9bf9-124fd35281a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:45.758716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82648c70-c4fc-4ee3-9bf9-124fd35281a3 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:45.772055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:45.782778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 691/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:47.996059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f44a7-e287-4582-9496-65c08e851c1c None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:47.996599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f44a7-e287-4582-9496-65c08e851c1c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:47.996781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f44a7-e287-4582-9496-65c08e851c1c None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:47.998250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 692/1383] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:48.004657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cbf58-f36b-484f-9970-df497c6933ce None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:48.005699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cbf58-f36b-484f-9970-df497c6933ce None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:48.005699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cbf58-f36b-484f-9970-df497c6933ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:48.075527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-cd6cbf58-f36b-484f-9970-df497c6933ce None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:48.076669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-cd6cbf58-f36b-484f-9970-df497c6933ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:48.175259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 692/1384] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:48.464224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f44a7-e287-4582-9496-65c08e851c1c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:48.465824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-707f44a7-e287-4582-9496-65c08e851c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:48.622988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f44a7-e287-4582-9496-65c08e851c1c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:48.693773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:48.740587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a58c53-2b19-4f56-bcce-03daf02c411e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:48.741194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58c53-2b19-4f56-bcce-03daf02c411e service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:48.741472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58c53-2b19-4f56-bcce-03daf02c411e service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:48.741737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58c53-2b19-4f56-bcce-03daf02c411e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:48.743421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-14a58c53-2b19-4f56-bcce-03daf02c411e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:48.847906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a58c53-2b19-4f56-bcce-03daf02c411e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:48.849508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a58c53-2b19-4f56-bcce-03daf02c411e service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:48.868341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 693/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:51.279163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a8e2-8472-4017-922f-8ae7fb0bf038 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:51.279842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a8e2-8472-4017-922f-8ae7fb0bf038 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:51.279842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a8e2-8472-4017-922f-8ae7fb0bf038 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:51.281113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 693/1386] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:51.286614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad4dfa-2e16-4eca-ac3b-881d560c7142 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:51.287069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad4dfa-2e16-4eca-ac3b-881d560c7142 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:51.287177 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad4dfa-2e16-4eca-ac3b-881d560c7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:51.358596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-91ad4dfa-2e16-4eca-ac3b-881d560c7142 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:51.359874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-91ad4dfa-2e16-4eca-ac3b-881d560c7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:51.396495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:51.465271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 694/1387] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:51.748874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12a8e2-8472-4017-922f-8ae7fb0bf038 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:51.750387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-db12a8e2-8472-4017-922f-8ae7fb0bf038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:51.829712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:51.901198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12a8e2-8472-4017-922f-8ae7fb0bf038 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:52.020640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20838b2f-a50e-4e2b-b726-4a274e4f959a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:52.021182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20838b2f-a50e-4e2b-b726-4a274e4f959a service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:52.021455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20838b2f-a50e-4e2b-b726-4a274e4f959a service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:52.021716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20838b2f-a50e-4e2b-b726-4a274e4f959a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:52.023323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-20838b2f-a50e-4e2b-b726-4a274e4f959a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:52.062416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:52.114337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20838b2f-a50e-4e2b-b726-4a274e4f959a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:52.116062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20838b2f-a50e-4e2b-b726-4a274e4f959a service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:52.134979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 694/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:52.810618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99ae4-fbf1-4fec-9d62-6adfc7436cbb None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:52.811245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99ae4-fbf1-4fec-9d62-6adfc7436cbb None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:52.811245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99ae4-fbf1-4fec-9d62-6adfc7436cbb None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:52.812501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 695/1389] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:52.817900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a42c-9781-41f8-92c2-8b8d01f8bb3e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:52.818332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a42c-9781-41f8-92c2-8b8d01f8bb3e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:52.818439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a42c-9781-41f8-92c2-8b8d01f8bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:52.887393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-02c2a42c-9781-41f8-92c2-8b8d01f8bb3e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:52.888650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-02c2a42c-9781-41f8-92c2-8b8d01f8bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:52.990319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 695/1390] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:53.287152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4355-9b77-494b-aa25-03b56b0cdbbf None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:53.287553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4355-9b77-494b-aa25-03b56b0cdbbf None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:53.287828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4355-9b77-494b-aa25-03b56b0cdbbf None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:53.289305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 696/1391] 10.222.0.44 () {54 vars in 962 bytes} [Tue Oct 4 08:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:53.294929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba0eba-a923-4025-88ca-b928df049073 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:53.295237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba0eba-a923-4025-88ca-b928df049073 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:53.295511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba0eba-a923-4025-88ca-b928df049073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:53.312509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:53.373665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-84ba0eba-a923-4025-88ca-b928df049073 None None] MFA Rules not processed for user `557974bcc89c412f8d5c2e2039843f65`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:53.374912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-84ba0eba-a923-4025-88ca-b928df049073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:53.460966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 696/1392] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Oct 4 08:44:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:53.471007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4355-9b77-494b-aa25-03b56b0cdbbf None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:53.472674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2cce4355-9b77-494b-aa25-03b56b0cdbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:53.593317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4355-9b77-494b-aa25-03b56b0cdbbf None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:53.657926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:53.674824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0251e-8a16-4fac-8f47-c1683d2acb74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:53.675450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0251e-8a16-4fac-8f47-c1683d2acb74 service glance] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:53.675731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0251e-8a16-4fac-8f47-c1683d2acb74 service glance] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:53.676044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0251e-8a16-4fac-8f47-c1683d2acb74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:53.677665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-95d0251e-8a16-4fac-8f47-c1683d2acb74 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:53.792864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0251e-8a16-4fac-8f47-c1683d2acb74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:53.794350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0251e-8a16-4fac-8f47-c1683d2acb74 service glance] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:53.812699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 697/1393] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:54.298255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b658d-19ce-47a5-9ba0-41d545eff3f8 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:54.298809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b658d-19ce-47a5-9ba0-41d545eff3f8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:54.298809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b658d-19ce-47a5-9ba0-41d545eff3f8 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:54.300266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 697/1394] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:54.305952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2a50b-2349-4143-9247-133b9d74d0fd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:54.306244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2a50b-2349-4143-9247-133b9d74d0fd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:54.306516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2a50b-2349-4143-9247-133b9d74d0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:54.374794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-76f2a50b-2349-4143-9247-133b9d74d0fd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:54.376037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-76f2a50b-2349-4143-9247-133b9d74d0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:54.425038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:54.496195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 698/1395] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:54.573761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:54.574091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:54.574385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:54.641173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:54.642383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:54.704457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:54.736390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 698/1396] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:44:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:54.785295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d78d-e0f6-47b2-98fe-6fc841c93df0 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:54.785580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d78d-e0f6-47b2-98fe-6fc841c93df0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:54.785846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d78d-e0f6-47b2-98fe-6fc841c93df0 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:54.787319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 699/1397] 10.222.0.44 () {54 vars in 943 bytes} [Tue Oct 4 08:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:54.791597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:54.792924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:54.793203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07b7c-040d-4c2c-8d1a-5819c9e339cd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:54.793487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07b7c-040d-4c2c-8d1a-5819c9e339cd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:54.793760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf07b7c-040d-4c2c-8d1a-5819c9e339cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:54.869334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3cf07b7c-040d-4c2c-8d1a-5819c9e339cd None None] MFA Rules not processed for user `c76ff74c7e67426bb2b94a4bb1a2b9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:54.870437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3cf07b7c-040d-4c2c-8d1a-5819c9e339cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:54.906433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9536b6-9272-4594-8831-15a8787e17a3 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:54.991607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 700/1398] 10.222.0.44 () {58 vars in 1040 bytes} [Tue Oct 4 08:44:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:44:54.996658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:55.001270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf07b7c-040d-4c2c-8d1a-5819c9e339cd None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:55.006494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:55.013398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf07b7c-040d-4c2c-8d1a-5819c9e339cd None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:55.033027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625561cf-e5db-46d7-84e6-7c4d4cead3ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:55.033654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625561cf-e5db-46d7-84e6-7c4d4cead3ea service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:55.033930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625561cf-e5db-46d7-84e6-7c4d4cead3ea service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:55.034251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625561cf-e5db-46d7-84e6-7c4d4cead3ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:55.035906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-625561cf-e5db-46d7-84e6-7c4d4cead3ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:55.132922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38863ffb-c3f7-4cf5-8a6c-4029df3d2197 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:55.133465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38863ffb-c3f7-4cf5-8a6c-4029df3d2197 service swift] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:55.133585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625561cf-e5db-46d7-84e6-7c4d4cead3ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:55.133759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38863ffb-c3f7-4cf5-8a6c-4029df3d2197 service swift] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:55.134008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38863ffb-c3f7-4cf5-8a6c-4029df3d2197 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:55.135092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625561cf-e5db-46d7-84e6-7c4d4cead3ea service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:55.135648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-38863ffb-c3f7-4cf5-8a6c-4029df3d2197 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:55.154337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 699/1399] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:55.228858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38863ffb-c3f7-4cf5-8a6c-4029df3d2197 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:55.230495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38863ffb-c3f7-4cf5-8a6c-4029df3d2197 service swift] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:55.240066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 701/1400] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:44:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:44:57.352176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793830b-6d01-4d11-a51a-acdaa9aa96c2 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:57.352743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793830b-6d01-4d11-a51a-acdaa9aa96c2 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:57.352743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793830b-6d01-4d11-a51a-acdaa9aa96c2 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:57.356469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 700/1401] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:44:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:57.359767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d11cb0-f101-4042-b344-f032d9402c5f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:57.360217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d11cb0-f101-4042-b344-f032d9402c5f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:57.360358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d11cb0-f101-4042-b344-f032d9402c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:57.402994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:57.438229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-58d11cb0-f101-4042-b344-f032d9402c5f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:57.439780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-58d11cb0-f101-4042-b344-f032d9402c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:57.554654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 702/1402] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:57.845393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793830b-6d01-4d11-a51a-acdaa9aa96c2 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:57.847370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5793830b-6d01-4d11-a51a-acdaa9aa96c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:57.942463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:58.009057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793830b-6d01-4d11-a51a-acdaa9aa96c2 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:58.130723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d6b02-3bfa-463e-aada-07fd1655bc74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:58.131915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d6b02-3bfa-463e-aada-07fd1655bc74 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:58.132253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d6b02-3bfa-463e-aada-07fd1655bc74 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:58.132644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d6b02-3bfa-463e-aada-07fd1655bc74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:58.134425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-915d6b02-3bfa-463e-aada-07fd1655bc74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:58.201653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:58.242945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d6b02-3bfa-463e-aada-07fd1655bc74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:58.244588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d6b02-3bfa-463e-aada-07fd1655bc74 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:58.264010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 701/1403] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:00.500521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb41ca2-35fd-40ce-acbb-1ca0638d91fa None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:00.501160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb41ca2-35fd-40ce-acbb-1ca0638d91fa None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:00.501160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb41ca2-35fd-40ce-acbb-1ca0638d91fa None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:00.502540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 703/1404] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:00.508482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888ccd-c5ce-40e6-8beb-e22f21b978b6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:00.508945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888ccd-c5ce-40e6-8beb-e22f21b978b6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:00.509052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90888ccd-c5ce-40e6-8beb-e22f21b978b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:00.704772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-90888ccd-c5ce-40e6-8beb-e22f21b978b6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:00.706506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-90888ccd-c5ce-40e6-8beb-e22f21b978b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:00.817332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 702/1405] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:01.101637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb41ca2-35fd-40ce-acbb-1ca0638d91fa None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:01.103187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-efb41ca2-35fd-40ce-acbb-1ca0638d91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:01.235160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:01.282629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb41ca2-35fd-40ce-acbb-1ca0638d91fa None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:01.437829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b233a-fedc-400d-8a26-41328428355c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:01.438448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b233a-fedc-400d-8a26-41328428355c service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:01.438716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b233a-fedc-400d-8a26-41328428355c service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:01.438990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b233a-fedc-400d-8a26-41328428355c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:01.440771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3a2b233a-fedc-400d-8a26-41328428355c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:01.545122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:01.567017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b233a-fedc-400d-8a26-41328428355c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:01.568646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b233a-fedc-400d-8a26-41328428355c service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:01.587201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 704/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:03.775783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10047c-19af-4eb1-baa4-a5330859a765 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:03.776368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10047c-19af-4eb1-baa4-a5330859a765 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:03.776368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10047c-19af-4eb1-baa4-a5330859a765 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:03.777854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 703/1407] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:03.783011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc97e17-0ea0-4cd5-a7c0-d671855cc0cf None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:03.783422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc97e17-0ea0-4cd5-a7c0-d671855cc0cf None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:03.783601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc97e17-0ea0-4cd5-a7c0-d671855cc0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:03.854689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2fc97e17-0ea0-4cd5-a7c0-d671855cc0cf None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:03.855938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2fc97e17-0ea0-4cd5-a7c0-d671855cc0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:04.028000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 705/1408] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:04.355227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10047c-19af-4eb1-baa4-a5330859a765 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:04.356796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fb10047c-19af-4eb1-baa4-a5330859a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:04.395822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:04.525498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10047c-19af-4eb1-baa4-a5330859a765 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:04.643247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:04.649919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc22c7-d1da-47fd-9f22-8b16b92537d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:04.650529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc22c7-d1da-47fd-9f22-8b16b92537d6 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:04.650799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc22c7-d1da-47fd-9f22-8b16b92537d6 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:04.651085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc22c7-d1da-47fd-9f22-8b16b92537d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:04.652704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-55dc22c7-d1da-47fd-9f22-8b16b92537d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:04.750472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc22c7-d1da-47fd-9f22-8b16b92537d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:04.752149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc22c7-d1da-47fd-9f22-8b16b92537d6 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:04.776754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 704/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:14.017564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=79 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:14.774058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=48 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:17.678668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c32811-5d39-4087-ae98-8a4cd4856829 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:17.679244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c32811-5d39-4087-ae98-8a4cd4856829 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:17.679244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c32811-5d39-4087-ae98-8a4cd4856829 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:17.680546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 706/1410] 10.222.0.44 () {54 vars in 918 bytes} [Tue Oct 4 08:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:17.686123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c282e71-b2d4-44a0-b6fa-ac675faac7c9 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:17.686561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c282e71-b2d4-44a0-b6fa-ac675faac7c9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:17.686675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c282e71-b2d4-44a0-b6fa-ac675faac7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:17.772990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0c282e71-b2d4-44a0-b6fa-ac675faac7c9 None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:17.774193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0c282e71-b2d4-44a0-b6fa-ac675faac7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:17.888123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 705/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Oct 4 08:45:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:17.902886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe44c1-72f4-4172-adc3-a68970b1209a None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:17.903268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe44c1-72f4-4172-adc3-a68970b1209a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:17.903655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe44c1-72f4-4172-adc3-a68970b1209a None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:17.905037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 707/1412] 10.222.0.44 () {54 vars in 918 bytes} [Tue Oct 4 08:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:17.910506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8df6de-af10-40e4-b8c5-a898553ccabf None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:17.910803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8df6de-af10-40e4-b8c5-a898553ccabf None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:17.911054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8df6de-af10-40e4-b8c5-a898553ccabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:17.986490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:17.989394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5c8df6de-af10-40e4-b8c5-a898553ccabf None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:17.991289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5c8df6de-af10-40e4-b8c5-a898553ccabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:18.113992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 706/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Oct 4 08:45:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:20.703672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ef52-e34a-4a25-9e80-081ace989b5d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:20.704556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ef52-e34a-4a25-9e80-081ace989b5d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:20.704556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ef52-e34a-4a25-9e80-081ace989b5d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:20.706668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:20.715817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc354-1113-4560-abdc-424706acd2c0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:20.716264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc354-1113-4560-abdc-424706acd2c0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:20.716406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc354-1113-4560-abdc-424706acd2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:20.795496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4dfcc354-1113-4560-abdc-424706acd2c0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:20.796754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4dfcc354-1113-4560-abdc-424706acd2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:20.854064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:20.904997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:21.193667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ef52-e34a-4a25-9e80-081ace989b5d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:21.195230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4842ef52-e34a-4a25-9e80-081ace989b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:21.364421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ef52-e34a-4a25-9e80-081ace989b5d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:21.443590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:21.485402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928af31-93a3-496f-ad36-e8c1f80cec13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:21.486010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928af31-93a3-496f-ad36-e8c1f80cec13 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:21.486283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928af31-93a3-496f-ad36-e8c1f80cec13 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:21.486555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928af31-93a3-496f-ad36-e8c1f80cec13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:21.488186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5928af31-93a3-496f-ad36-e8c1f80cec13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:21.585044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928af31-93a3-496f-ad36-e8c1f80cec13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:21.586671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928af31-93a3-496f-ad36-e8c1f80cec13 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:21.606875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 709/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:22.742249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea963d-f145-4173-9c52-3b8a875255c9 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:22.742852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea963d-f145-4173-9c52-3b8a875255c9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:22.742852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea963d-f145-4173-9c52-3b8a875255c9 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:22.744174 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:22.751534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4bf4f-c424-4ee1-a525-d288441c31bc None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:22.752621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4bf4f-c424-4ee1-a525-d288441c31bc None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:22.752825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4bf4f-c424-4ee1-a525-d288441c31bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:22.844172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c9a4bf4f-c424-4ee1-a525-d288441c31bc None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:22.845401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c9a4bf4f-c424-4ee1-a525-d288441c31bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:22.873364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:22.950490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:23.101979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea963d-f145-4173-9c52-3b8a875255c9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:23.104418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-93ea963d-f145-4173-9c52-3b8a875255c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:23.265894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea963d-f145-4173-9c52-3b8a875255c9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:23.308511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:23.394740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bd54e-d578-439a-81f9-8daed0254f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:23.395449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bd54e-d578-439a-81f9-8daed0254f07 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:23.395722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bd54e-d578-439a-81f9-8daed0254f07 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:23.396080 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bd54e-d578-439a-81f9-8daed0254f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:23.397738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-188bd54e-d578-439a-81f9-8daed0254f07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:23.489423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bd54e-d578-439a-81f9-8daed0254f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:23.491016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bd54e-d578-439a-81f9-8daed0254f07 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:23.510715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 709/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:24.635589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2fcdc-900a-4a2f-a74a-c360106f9b87 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:24.636192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2fcdc-900a-4a2f-a74a-c360106f9b87 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:24.636192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2fcdc-900a-4a2f-a74a-c360106f9b87 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:24.637528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:24.644068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cd85b-1add-4eb2-8242-9af8db537283 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:24.644593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cd85b-1add-4eb2-8242-9af8db537283 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:24.644699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cd85b-1add-4eb2-8242-9af8db537283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:24.707229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:24.731527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-566cd85b-1add-4eb2-8242-9af8db537283 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:24.732781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-566cd85b-1add-4eb2-8242-9af8db537283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:24.849328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:25.005100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2fcdc-900a-4a2f-a74a-c360106f9b87 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:25.006660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1aa2fcdc-900a-4a2f-a74a-c360106f9b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:25.186056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2fcdc-900a-4a2f-a74a-c360106f9b87 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:25.198968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:25.310685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1e7de-3fd9-445d-b860-5b10c744b762 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:25.311287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1e7de-3fd9-445d-b860-5b10c744b762 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:25.311557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1e7de-3fd9-445d-b860-5b10c744b762 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:25.311827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1e7de-3fd9-445d-b860-5b10c744b762 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:25.313440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-07a1e7de-3fd9-445d-b860-5b10c744b762 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:25.408750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1e7de-3fd9-445d-b860-5b10c744b762 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:25.410337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1e7de-3fd9-445d-b860-5b10c744b762 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:25.428871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 712/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:26.551308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c88f-ba40-40e4-9157-e453eca86be9 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:26.551939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c88f-ba40-40e4-9157-e453eca86be9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:26.551939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c88f-ba40-40e4-9157-e453eca86be9 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:26.553378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:26.560917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21cb79-1d3e-4488-8cd6-3e00fc39b126 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:26.561349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21cb79-1d3e-4488-8cd6-3e00fc39b126 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:26.561503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21cb79-1d3e-4488-8cd6-3e00fc39b126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:26.584450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:26.644509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ba21cb79-1d3e-4488-8cd6-3e00fc39b126 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:26.645746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ba21cb79-1d3e-4488-8cd6-3e00fc39b126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:26.747579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:26.899274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02c88f-ba40-40e4-9157-e453eca86be9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:26.900768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1b02c88f-ba40-40e4-9157-e453eca86be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:27.033356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:27.070176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02c88f-ba40-40e4-9157-e453eca86be9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:27.208522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ebaa6-50ca-4442-8157-65145a9245c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:27.209040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ebaa6-50ca-4442-8157-65145a9245c4 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:27.209315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ebaa6-50ca-4442-8157-65145a9245c4 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:27.209588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ebaa6-50ca-4442-8157-65145a9245c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:27.211198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-df6ebaa6-50ca-4442-8157-65145a9245c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:27.291730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:27.305615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ebaa6-50ca-4442-8157-65145a9245c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:27.307213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ebaa6-50ca-4442-8157-65145a9245c4 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:27.325232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 712/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:28.454972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc05a-2b20-4f2c-9d57-6ba93d7e28de None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:28.455842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc05a-2b20-4f2c-9d57-6ba93d7e28de None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:28.455842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcc05a-2b20-4f2c-9d57-6ba93d7e28de None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:28.458078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:28.464845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa572f6-48e6-429d-8477-a42cc4187798 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:28.465315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa572f6-48e6-429d-8477-a42cc4187798 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:28.465417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa572f6-48e6-429d-8477-a42cc4187798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:28.543901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-caa572f6-48e6-429d-8477-a42cc4187798 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:28.545125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-caa572f6-48e6-429d-8477-a42cc4187798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:28.648386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:28.818223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc05a-2b20-4f2c-9d57-6ba93d7e28de None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:28.820268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-22fcc05a-2b20-4f2c-9d57-6ba93d7e28de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:28.908920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:28.973016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcc05a-2b20-4f2c-9d57-6ba93d7e28de None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:29.090621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef50903-9cb9-4081-8234-50bb68fc40b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:29.091229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef50903-9cb9-4081-8234-50bb68fc40b2 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:29.091503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef50903-9cb9-4081-8234-50bb68fc40b2 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:29.091795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef50903-9cb9-4081-8234-50bb68fc40b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:29.093393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7ef50903-9cb9-4081-8234-50bb68fc40b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:29.144110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:29.186696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef50903-9cb9-4081-8234-50bb68fc40b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:29.188237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef50903-9cb9-4081-8234-50bb68fc40b2 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:29.206178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 715/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:30.254224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49120692-aa73-4a09-9d3f-0c62b38c2f5b None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:30.254920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49120692-aa73-4a09-9d3f-0c62b38c2f5b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:30.254920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49120692-aa73-4a09-9d3f-0c62b38c2f5b None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:30.256231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:30.262749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1db9-bcd4-497d-b0bc-73cab9af228b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:30.263230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1db9-bcd4-497d-b0bc-73cab9af228b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:30.263331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1db9-bcd4-497d-b0bc-73cab9af228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:30.343170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-139e1db9-bcd4-497d-b0bc-73cab9af228b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:30.344454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-139e1db9-bcd4-497d-b0bc-73cab9af228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:30.444250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:30.604090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49120692-aa73-4a09-9d3f-0c62b38c2f5b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:30.606900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-49120692-aa73-4a09-9d3f-0c62b38c2f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:30.659810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:30.767436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49120692-aa73-4a09-9d3f-0c62b38c2f5b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:30.888708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582a216-a10c-4c7d-9270-b71596e6b93b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:30.889228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582a216-a10c-4c7d-9270-b71596e6b93b service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:30.889506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582a216-a10c-4c7d-9270-b71596e6b93b service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:30.889776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582a216-a10c-4c7d-9270-b71596e6b93b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:30.891373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6582a216-a10c-4c7d-9270-b71596e6b93b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:30.898410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:30.987332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582a216-a10c-4c7d-9270-b71596e6b93b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:30.988990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582a216-a10c-4c7d-9270-b71596e6b93b service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:31.009302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 715/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:32.087432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ea2f-cb24-48e1-9560-bccc876d8510 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:32.088402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ea2f-cb24-48e1-9560-bccc876d8510 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:32.088402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ea2f-cb24-48e1-9560-bccc876d8510 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:32.090536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:32.097780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cec8bb-d97f-427f-b5ef-08ff13d7e45a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:32.098513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cec8bb-d97f-427f-b5ef-08ff13d7e45a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:32.098679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cec8bb-d97f-427f-b5ef-08ff13d7e45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:32.182746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-50cec8bb-d97f-427f-b5ef-08ff13d7e45a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:32.183975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-50cec8bb-d97f-427f-b5ef-08ff13d7e45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:32.254661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:32.286347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:32.437275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049ea2f-cb24-48e1-9560-bccc876d8510 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:32.438822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2049ea2f-cb24-48e1-9560-bccc876d8510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:32.456922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:32.590368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049ea2f-cb24-48e1-9560-bccc876d8510 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:32.691012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:32.718699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679b4fe-0df9-40e3-88f2-e0a459756a37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:32.719313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679b4fe-0df9-40e3-88f2-e0a459756a37 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:32.719606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679b4fe-0df9-40e3-88f2-e0a459756a37 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:32.719857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679b4fe-0df9-40e3-88f2-e0a459756a37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:32.721475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a679b4fe-0df9-40e3-88f2-e0a459756a37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:32.823930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679b4fe-0df9-40e3-88f2-e0a459756a37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:32.825542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679b4fe-0df9-40e3-88f2-e0a459756a37 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:32.845427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 718/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:33.896781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffed6ad-0327-44d8-9fa8-e71a9a065c44 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:33.897721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffed6ad-0327-44d8-9fa8-e71a9a065c44 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:33.897721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffed6ad-0327-44d8-9fa8-e71a9a065c44 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:33.899796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:33.907225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247bf3-29c8-4c8d-9f50-4eb02ef34ac1 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:33.908022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247bf3-29c8-4c8d-9f50-4eb02ef34ac1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:33.908210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247bf3-29c8-4c8d-9f50-4eb02ef34ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:33.992277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4e247bf3-29c8-4c8d-9f50-4eb02ef34ac1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:33.993515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4e247bf3-29c8-4c8d-9f50-4eb02ef34ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.036316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:34.097480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.260993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffed6ad-0327-44d8-9fa8-e71a9a065c44 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.262451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-cffed6ad-0327-44d8-9fa8-e71a9a065c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.411728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffed6ad-0327-44d8-9fa8-e71a9a065c44 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.479408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:34.541793 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42615d9e-590e-4c1b-9e1b-aa5553e127db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.542422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42615d9e-590e-4c1b-9e1b-aa5553e127db service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.542733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42615d9e-590e-4c1b-9e1b-aa5553e127db service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.543010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42615d9e-590e-4c1b-9e1b-aa5553e127db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.544817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-42615d9e-590e-4c1b-9e1b-aa5553e127db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.640362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42615d9e-590e-4c1b-9e1b-aa5553e127db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.641979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42615d9e-590e-4c1b-9e1b-aa5553e127db service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.661313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 718/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:35.735730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752da8e-76e2-4f00-9b2c-654e26b692ec None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.736354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752da8e-76e2-4f00-9b2c-654e26b692ec None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.736354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752da8e-76e2-4f00-9b2c-654e26b692ec None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.737698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:35.744400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f45962-0249-4264-8bf0-7d7290696162 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.744887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f45962-0249-4264-8bf0-7d7290696162 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.744887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f45962-0249-4264-8bf0-7d7290696162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.831116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-11f45962-0249-4264-8bf0-7d7290696162 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:35.832366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-11f45962-0249-4264-8bf0-7d7290696162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:35.834458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:35.928926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.084954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752da8e-76e2-4f00-9b2c-654e26b692ec None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.086586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c752da8e-76e2-4f00-9b2c-654e26b692ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.251969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752da8e-76e2-4f00-9b2c-654e26b692ec None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.290985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:36.384247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184728be-df0a-44bb-b950-e36bf1ab1215 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.384867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184728be-df0a-44bb-b950-e36bf1ab1215 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.385147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184728be-df0a-44bb-b950-e36bf1ab1215 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.385417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184728be-df0a-44bb-b950-e36bf1ab1215 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.387047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-184728be-df0a-44bb-b950-e36bf1ab1215 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.481265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184728be-df0a-44bb-b950-e36bf1ab1215 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.482865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184728be-df0a-44bb-b950-e36bf1ab1215 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.501731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 721/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:38.550804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21fe6-cfd7-4e97-b2b7-2d49a8d2e1a5 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:38.551406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21fe6-cfd7-4e97-b2b7-2d49a8d2e1a5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:38.551406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f21fe6-cfd7-4e97-b2b7-2d49a8d2e1a5 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:38.552758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:38.559274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4d526-b906-4f88-9906-38b8f0fb8554 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:38.560026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4d526-b906-4f88-9906-38b8f0fb8554 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:38.560214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4d526-b906-4f88-9906-38b8f0fb8554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:38.608274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:38.642694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e8f4d526-b906-4f88-9906-38b8f0fb8554 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:38.643922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e8f4d526-b906-4f88-9906-38b8f0fb8554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:38.750764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:39.044708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21fe6-cfd7-4e97-b2b7-2d49a8d2e1a5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:39.047126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-29f21fe6-cfd7-4e97-b2b7-2d49a8d2e1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:39.184804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f21fe6-cfd7-4e97-b2b7-2d49a8d2e1a5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:39.220504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:39.268596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7568f-9e31-4c35-a629-634037082800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:39.269180 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7568f-9e31-4c35-a629-634037082800 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:39.269452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7568f-9e31-4c35-a629-634037082800 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:39.269717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7568f-9e31-4c35-a629-634037082800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:39.271313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a7d7568f-9e31-4c35-a629-634037082800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:39.384189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7568f-9e31-4c35-a629-634037082800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:39.385651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7568f-9e31-4c35-a629-634037082800 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:39.403519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 721/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:41.721635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab1d73-318b-4930-9344-d9ca09e3f000 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:41.722171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab1d73-318b-4930-9344-d9ca09e3f000 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:41.722171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab1d73-318b-4930-9344-d9ca09e3f000 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:41.724294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:41.733215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acaedc-81b5-490a-bf98-64a42c6e285a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:41.733613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acaedc-81b5-490a-bf98-64a42c6e285a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:41.733819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acaedc-81b5-490a-bf98-64a42c6e285a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:41.805001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-07acaedc-81b5-490a-bf98-64a42c6e285a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:41.806216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-07acaedc-81b5-490a-bf98-64a42c6e285a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:41.819186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:41.907598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:42.191558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab1d73-318b-4930-9344-d9ca09e3f000 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:42.193939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5cab1d73-318b-4930-9344-d9ca09e3f000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:42.317150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab1d73-318b-4930-9344-d9ca09e3f000 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:42.330418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:42.400694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7489517-44d2-4376-a80a-e805e91d8a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:42.401312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489517-44d2-4376-a80a-e805e91d8a42 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.401591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489517-44d2-4376-a80a-e805e91d8a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.401866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489517-44d2-4376-a80a-e805e91d8a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.403484 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e7489517-44d2-4376-a80a-e805e91d8a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:42.520300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7489517-44d2-4376-a80a-e805e91d8a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:42.521742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7489517-44d2-4376-a80a-e805e91d8a42 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:42.549183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 724/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:45.079674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119b599-f499-42c8-82fa-ad5e9b7168f4 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:45.080235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119b599-f499-42c8-82fa-ad5e9b7168f4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:45.080235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119b599-f499-42c8-82fa-ad5e9b7168f4 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:45.081629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:45.087845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f83d74-3629-43e3-a9eb-40571498c603 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:45.088566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f83d74-3629-43e3-a9eb-40571498c603 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:45.088688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f83d74-3629-43e3-a9eb-40571498c603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:45.148998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:45.168039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-66f83d74-3629-43e3-a9eb-40571498c603 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:45.169283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-66f83d74-3629-43e3-a9eb-40571498c603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:45.275331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:45.556924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119b599-f499-42c8-82fa-ad5e9b7168f4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:45.558520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a119b599-f499-42c8-82fa-ad5e9b7168f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:45.677201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119b599-f499-42c8-82fa-ad5e9b7168f4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:45.717995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:45.754063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec052-d5dc-46d2-907d-af08e34feb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:45.754672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec052-d5dc-46d2-907d-af08e34feb08 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:45.754943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec052-d5dc-46d2-907d-af08e34feb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:45.755208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec052-d5dc-46d2-907d-af08e34feb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:45.756846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-66bec052-d5dc-46d2-907d-af08e34feb08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:45.878035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bec052-d5dc-46d2-907d-af08e34feb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:45.879437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bec052-d5dc-46d2-907d-af08e34feb08 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:45.899009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 724/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:48.182617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb071-f8ce-46ba-b664-1475f58c9d3b None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:48.183180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb071-f8ce-46ba-b664-1475f58c9d3b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:48.183180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb071-f8ce-46ba-b664-1475f58c9d3b None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:48.184540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:48.191005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122c853-4bf4-4644-b51c-a193aaaac4ef None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:48.191788 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122c853-4bf4-4644-b51c-a193aaaac4ef None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:48.191994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122c853-4bf4-4644-b51c-a193aaaac4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:48.276089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5122c853-4bf4-4644-b51c-a193aaaac4ef None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:48.277303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5122c853-4bf4-4644-b51c-a193aaaac4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:48.298692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:48.375869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:48.658285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbb071-f8ce-46ba-b664-1475f58c9d3b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:48.660874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-92dbb071-f8ce-46ba-b664-1475f58c9d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:48.770373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbb071-f8ce-46ba-b664-1475f58c9d3b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:48.791375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:48.851278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6494b-5aeb-476d-ab9b-364ce89fc116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:48.851913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6494b-5aeb-476d-ab9b-364ce89fc116 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:48.852207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6494b-5aeb-476d-ab9b-364ce89fc116 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:48.852478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6494b-5aeb-476d-ab9b-364ce89fc116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:48.854086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-93e6494b-5aeb-476d-ab9b-364ce89fc116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:48.975378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6494b-5aeb-476d-ab9b-364ce89fc116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:48.976844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6494b-5aeb-476d-ab9b-364ce89fc116 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:48.995579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 727/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:51.270445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddba16-6873-41b8-87e1-63e1eff953ff None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.271298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddba16-6873-41b8-87e1-63e1eff953ff None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.271298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddba16-6873-41b8-87e1-63e1eff953ff None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.273430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:51.279134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ce13-b984-4432-b5f1-faeda523fecd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.279597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ce13-b984-4432-b5f1-faeda523fecd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.279713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ce13-b984-4432-b5f1-faeda523fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.344453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:51.347870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-bb46ce13-b984-4432-b5f1-faeda523fecd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:51.350032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bb46ce13-b984-4432-b5f1-faeda523fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.462609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:51.751846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddba16-6873-41b8-87e1-63e1eff953ff None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:51.754516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-dbddba16-6873-41b8-87e1-63e1eff953ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.867429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddba16-6873-41b8-87e1-63e1eff953ff None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:51.922335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:51.951621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192bf8d-c2f8-4e30-90c1-61989eef3273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:51.952264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192bf8d-c2f8-4e30-90c1-61989eef3273 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.952539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192bf8d-c2f8-4e30-90c1-61989eef3273 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.952805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192bf8d-c2f8-4e30-90c1-61989eef3273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.954409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7192bf8d-c2f8-4e30-90c1-61989eef3273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:52.078723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192bf8d-c2f8-4e30-90c1-61989eef3273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:52.080205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192bf8d-c2f8-4e30-90c1-61989eef3273 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:52.100208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 727/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:54.297440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb02b1-f29b-48fc-b235-b6ee9d19a24d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.298020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb02b1-f29b-48fc-b235-b6ee9d19a24d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.298020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb02b1-f29b-48fc-b235-b6ee9d19a24d None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.299330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:54.305934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090367c-a99c-4970-a6c8-ab40a666b223 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.306662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090367c-a99c-4970-a6c8-ab40a666b223 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.306868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090367c-a99c-4970-a6c8-ab40a666b223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.380736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0090367c-a99c-4970-a6c8-ab40a666b223 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:54.381934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0090367c-a99c-4970-a6c8-ab40a666b223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:54.410797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:54.479332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:54.769571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb02b1-f29b-48fc-b235-b6ee9d19a24d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:54.771870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-06fb02b1-f29b-48fc-b235-b6ee9d19a24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:54.886846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb02b1-f29b-48fc-b235-b6ee9d19a24d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:54.918628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:54.977464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657efa4-1a06-40dd-836f-18661758b27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:54.978082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657efa4-1a06-40dd-836f-18661758b27c service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.978384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657efa4-1a06-40dd-836f-18661758b27c service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.978653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657efa4-1a06-40dd-836f-18661758b27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.980402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4657efa4-1a06-40dd-836f-18661758b27c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:55.098155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657efa4-1a06-40dd-836f-18661758b27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:55.099679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657efa4-1a06-40dd-836f-18661758b27c service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:55.118673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 730/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:57.310604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd96ef-4258-4d69-97ce-17129f819363 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:57.311199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd96ef-4258-4d69-97ce-17129f819363 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:57.311199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd96ef-4258-4d69-97ce-17129f819363 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:57.312653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:57.319342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ce69d-43e2-4b22-846f-b2cf686e15d4 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:57.320053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ce69d-43e2-4b22-846f-b2cf686e15d4 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:57.320211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ce69d-43e2-4b22-846f-b2cf686e15d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:57.402362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-171ce69d-43e2-4b22-846f-b2cf686e15d4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:57.403579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-171ce69d-43e2-4b22-846f-b2cf686e15d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:57.412201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:57.503852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:57.791825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd96ef-4258-4d69-97ce-17129f819363 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:57.794315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9dcd96ef-4258-4d69-97ce-17129f819363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:57.907302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd96ef-4258-4d69-97ce-17129f819363 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:57.971817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:57.996428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3174cc-9322-4371-bfdb-88b18d03a514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:57.997048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3174cc-9322-4371-bfdb-88b18d03a514 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:57.997322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3174cc-9322-4371-bfdb-88b18d03a514 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:57.997587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3174cc-9322-4371-bfdb-88b18d03a514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:58.001796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fb3174cc-9322-4371-bfdb-88b18d03a514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:58.118522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3174cc-9322-4371-bfdb-88b18d03a514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:58.119927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3174cc-9322-4371-bfdb-88b18d03a514 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:58.139525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 730/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:00.304885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd07ba-39e4-4f31-9a1a-483adc7a7fb1 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.305723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd07ba-39e4-4f31-9a1a-483adc7a7fb1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.305723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd07ba-39e4-4f31-9a1a-483adc7a7fb1 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.307921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:46:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:00.313439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dfb90-a49f-4b09-9556-8237f19a4d02 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.313880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dfb90-a49f-4b09-9556-8237f19a4d02 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.313997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dfb90-a49f-4b09-9556-8237f19a4d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.381066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-973dfb90-a49f-4b09-9556-8237f19a4d02 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:00.382433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-973dfb90-a49f-4b09-9556-8237f19a4d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:00.423007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:00.483368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:00.762861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd07ba-39e4-4f31-9a1a-483adc7a7fb1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:00.765420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3bfd07ba-39e4-4f31-9a1a-483adc7a7fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:00.886330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd07ba-39e4-4f31-9a1a-483adc7a7fb1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:00.935307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:00.974481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409ec8-0381-4fc0-ac2a-11e6bb49a453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:00.975101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409ec8-0381-4fc0-ac2a-11e6bb49a453 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.975386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409ec8-0381-4fc0-ac2a-11e6bb49a453 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.975660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409ec8-0381-4fc0-ac2a-11e6bb49a453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.977299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-86409ec8-0381-4fc0-ac2a-11e6bb49a453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:01.093466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409ec8-0381-4fc0-ac2a-11e6bb49a453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:01.094905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409ec8-0381-4fc0-ac2a-11e6bb49a453 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:01.114775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 733/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:03.285028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd24be-4a8e-46c1-be3b-717cd765a3bc None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.285619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd24be-4a8e-46c1-be3b-717cd765a3bc None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.285619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd24be-4a8e-46c1-be3b-717cd765a3bc None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.286970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:03.292606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297146d-c5de-49fe-9b7d-4bef0804e702 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.293036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297146d-c5de-49fe-9b7d-4bef0804e702 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.293153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297146d-c5de-49fe-9b7d-4bef0804e702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.366175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e297146d-c5de-49fe-9b7d-4bef0804e702 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:03.367382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e297146d-c5de-49fe-9b7d-4bef0804e702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:03.386392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:03.474005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:03.754617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd24be-4a8e-46c1-be3b-717cd765a3bc None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.757128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-72cd24be-4a8e-46c1-be3b-717cd765a3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:03.870410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd24be-4a8e-46c1-be3b-717cd765a3bc None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.942905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:03.950075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c3ed7-b97f-4d97-a622-caae19c8782d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.950702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c3ed7-b97f-4d97-a622-caae19c8782d service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.950978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c3ed7-b97f-4d97-a622-caae19c8782d service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.951249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c3ed7-b97f-4d97-a622-caae19c8782d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.953243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-be7c3ed7-b97f-4d97-a622-caae19c8782d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:04.076919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c3ed7-b97f-4d97-a622-caae19c8782d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:04.078380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c3ed7-b97f-4d97-a622-caae19c8782d service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:04.097689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 733/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:06.255394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95b85-a621-45e4-b32d-9742018e3070 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:06.256362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95b85-a621-45e4-b32d-9742018e3070 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:06.256362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95b85-a621-45e4-b32d-9742018e3070 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:06.258288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:46:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:06.267277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fbaa8-b0aa-4918-90bb-b340cfd119c1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:06.267702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fbaa8-b0aa-4918-90bb-b340cfd119c1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:06.267946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fbaa8-b0aa-4918-90bb-b340cfd119c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:06.342390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-1c5fbaa8-b0aa-4918-90bb-b340cfd119c1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:06.343608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1c5fbaa8-b0aa-4918-90bb-b340cfd119c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:06.395597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:06.452999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:06.740823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae95b85-a621-45e4-b32d-9742018e3070 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:06.742474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6ae95b85-a621-45e4-b32d-9742018e3070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:06.863738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae95b85-a621-45e4-b32d-9742018e3070 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:06.916025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:06.951232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46c240-70f4-4cf2-a2c4-027a8965dcdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:06.951853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46c240-70f4-4cf2-a2c4-027a8965dcdd service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:06.952170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46c240-70f4-4cf2-a2c4-027a8965dcdd service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:06.952481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46c240-70f4-4cf2-a2c4-027a8965dcdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:06.954097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2e46c240-70f4-4cf2-a2c4-027a8965dcdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:07.076130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46c240-70f4-4cf2-a2c4-027a8965dcdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:07.077581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46c240-70f4-4cf2-a2c4-027a8965dcdd service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:07.097184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 736/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:16.450286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=22 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:17.094570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=66 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:26.855656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e31d7a-d1e6-475f-b416-a6b8e3f491a0 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:26.856279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e31d7a-d1e6-475f-b416-a6b8e3f491a0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:26.856279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e31d7a-d1e6-475f-b416-a6b8e3f491a0 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:26.858381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 735/1471] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:46:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:26.865427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89bd92-8b7f-4f03-9d9e-274b2c7fce6b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:26.865920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89bd92-8b7f-4f03-9d9e-274b2c7fce6b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:26.865920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89bd92-8b7f-4f03-9d9e-274b2c7fce6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:26.935873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-df89bd92-8b7f-4f03-9d9e-274b2c7fce6b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:26.937130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-df89bd92-8b7f-4f03-9d9e-274b2c7fce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:27.052842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 737/1472] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:46:27.367029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b129c-5816-4d7b-9567-5c7004d72743 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:27.367388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b129c-5816-4d7b-9567-5c7004d72743 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:27.367852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b129c-5816-4d7b-9567-5c7004d72743 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:27.369224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 736/1473] 10.222.0.44 () {54 vars in 962 bytes} [Tue Oct 4 08:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:27.374819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec375d7e-6384-44a5-9626-0a3273082d28 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:27.375120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec375d7e-6384-44a5-9626-0a3273082d28 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:27.375384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec375d7e-6384-44a5-9626-0a3273082d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:27.458226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ec375d7e-6384-44a5-9626-0a3273082d28 None None] MFA Rules not processed for user `557974bcc89c412f8d5c2e2039843f65`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:27.459461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ec375d7e-6384-44a5-9626-0a3273082d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:27.481423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:27.543581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 738/1474] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Oct 4 08:46:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:27.551709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b129c-5816-4d7b-9567-5c7004d72743 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:27.553269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-328b129c-5816-4d7b-9567-5c7004d72743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:27.670948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b129c-5816-4d7b-9567-5c7004d72743 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:27.747918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e9a8c-be87-4008-aed0-af19a9d009e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:27.748518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e9a8c-be87-4008-aed0-af19a9d009e2 service glance] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:27.748793 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e9a8c-be87-4008-aed0-af19a9d009e2 service glance] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:27.749059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e9a8c-be87-4008-aed0-af19a9d009e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:27.750647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-891e9a8c-be87-4008-aed0-af19a9d009e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:27.808570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:27.868040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e9a8c-be87-4008-aed0-af19a9d009e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:27.869474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e9a8c-be87-4008-aed0-af19a9d009e2 service glance] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:27.888973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 737/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:30.117959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8f63-1a6b-46e9-8940-a59273dd99fb None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:30.118828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8f63-1a6b-46e9-8940-a59273dd99fb None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:30.118828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a8f63-1a6b-46e9-8940-a59273dd99fb None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:30.120992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 739/1476] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:46:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:30.127766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4e096-9b50-4b74-8160-883ccf3f0ebe None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:30.128273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4e096-9b50-4b74-8160-883ccf3f0ebe None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:30.128273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4e096-9b50-4b74-8160-883ccf3f0ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:30.195651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-03e4e096-9b50-4b74-8160-883ccf3f0ebe None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:30.196880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-03e4e096-9b50-4b74-8160-883ccf3f0ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:30.300325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 738/1477] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:30.576595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a8f63-1a6b-46e9-8940-a59273dd99fb None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:30.579066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-710a8f63-1a6b-46e9-8940-a59273dd99fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:30.728096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a8f63-1a6b-46e9-8940-a59273dd99fb None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:30.779313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:30.825386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b32856-98ed-4f5f-9f9b-e80172d26589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:30.825986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b32856-98ed-4f5f-9f9b-e80172d26589 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:30.826261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b32856-98ed-4f5f-9f9b-e80172d26589 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:30.826525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b32856-98ed-4f5f-9f9b-e80172d26589 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:30.827217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b32856-98ed-4f5f-9f9b-e80172d26589 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:30.828906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-a6b32856-98ed-4f5f-9f9b-e80172d26589 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:46:30.828906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:46:30.828906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:46:30.830035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b32856-98ed-4f5f-9f9b-e80172d26589 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:30.850920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 740/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Oct 4 08:46:30 2022] GET /identity/v3/projects => generated 2553 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:32.991302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65742b9f-96ef-420e-887e-58d69e05209b None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:32.991824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65742b9f-96ef-420e-887e-58d69e05209b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:32.991974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65742b9f-96ef-420e-887e-58d69e05209b None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:32.993236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 739/1479] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:32.999194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445f5c9-a40d-4d5c-b65a-68608a54c775 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:32.999610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445f5c9-a40d-4d5c-b65a-68608a54c775 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:32.999738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445f5c9-a40d-4d5c-b65a-68608a54c775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.086078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2445f5c9-a40d-4d5c-b65a-68608a54c775 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:33.087936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2445f5c9-a40d-4d5c-b65a-68608a54c775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:33.197856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 741/1480] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:33.500088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65742b9f-96ef-420e-887e-58d69e05209b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.501855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-65742b9f-96ef-420e-887e-58d69e05209b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:33.512236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:33.668212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65742b9f-96ef-420e-887e-58d69e05209b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.753435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:33.786329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db927424-fe2b-40ea-b433-ee19ca8dd7b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.786927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db927424-fe2b-40ea-b433-ee19ca8dd7b5 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.787204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db927424-fe2b-40ea-b433-ee19ca8dd7b5 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.787473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db927424-fe2b-40ea-b433-ee19ca8dd7b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.789123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-db927424-fe2b-40ea-b433-ee19ca8dd7b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:33.882854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db927424-fe2b-40ea-b433-ee19ca8dd7b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.884495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db927424-fe2b-40ea-b433-ee19ca8dd7b5 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.903474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 740/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:36.222225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324c13d-b648-4f8e-a7c7-459b1bd5d431 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.223020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324c13d-b648-4f8e-a7c7-459b1bd5d431 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.223020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324c13d-b648-4f8e-a7c7-459b1bd5d431 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.225215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 742/1482] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:46:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:36.230911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f9187-935e-4438-984e-ed04c77cb89b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.231349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f9187-935e-4438-984e-ed04c77cb89b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.231445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f9187-935e-4438-984e-ed04c77cb89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.306467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ea4f9187-935e-4438-984e-ed04c77cb89b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:36.308223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ea4f9187-935e-4438-984e-ed04c77cb89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.364396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:36.420335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 741/1483] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:36.708167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324c13d-b648-4f8e-a7c7-459b1bd5d431 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:36.710585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c324c13d-b648-4f8e-a7c7-459b1bd5d431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.744715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:36.828339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324c13d-b648-4f8e-a7c7-459b1bd5d431 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:36.910674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41685c62-da50-4277-9bc1-c0ba8d173e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:36.911277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41685c62-da50-4277-9bc1-c0ba8d173e3c service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.911549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41685c62-da50-4277-9bc1-c0ba8d173e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.911819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41685c62-da50-4277-9bc1-c0ba8d173e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.913440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-41685c62-da50-4277-9bc1-c0ba8d173e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.992793 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:37.029587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41685c62-da50-4277-9bc1-c0ba8d173e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.031008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41685c62-da50-4277-9bc1-c0ba8d173e3c service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.050295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 743/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:39.247135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcd882-75f0-4559-85f4-eccf92f41428 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.248019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcd882-75f0-4559-85f4-eccf92f41428 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.248019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcd882-75f0-4559-85f4-eccf92f41428 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.249995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 742/1485] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:46:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:39.256365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2db0-4a2a-49e5-ba6d-e16475c04aae None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.256773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2db0-4a2a-49e5-ba6d-e16475c04aae None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.257030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2db0-4a2a-49e5-ba6d-e16475c04aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.330571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-423a2db0-4a2a-49e5-ba6d-e16475c04aae None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:39.331789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-423a2db0-4a2a-49e5-ba6d-e16475c04aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.438530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 744/1486] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:39.732281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcd882-75f0-4559-85f4-eccf92f41428 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.734682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-22bcd882-75f0-4559-85f4-eccf92f41428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.845212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcd882-75f0-4559-85f4-eccf92f41428 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.921578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:39.923111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd70fa-5a7d-4cfe-b9c1-a3a65e460315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.923644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd70fa-5a7d-4cfe-b9c1-a3a65e460315 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.923984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd70fa-5a7d-4cfe-b9c1-a3a65e460315 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.924252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd70fa-5a7d-4cfe-b9c1-a3a65e460315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.926508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d2cd70fa-5a7d-4cfe-b9c1-a3a65e460315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:40.046743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd70fa-5a7d-4cfe-b9c1-a3a65e460315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:40.048217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd70fa-5a7d-4cfe-b9c1-a3a65e460315 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:40.067424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 743/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:42.300533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4bcc-a91d-4f65-84c0-72c66715e37f None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:42.301104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4bcc-a91d-4f65-84c0-72c66715e37f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:42.301104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4bcc-a91d-4f65-84c0-72c66715e37f None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:42.302427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 745/1488] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:42.311593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7afae-a12d-414e-87bd-a546b109c398 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:42.312389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7afae-a12d-414e-87bd-a546b109c398 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:42.312526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7afae-a12d-414e-87bd-a546b109c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:42.389316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-68a7afae-a12d-414e-87bd-a546b109c398 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:42.390586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-68a7afae-a12d-414e-87bd-a546b109c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:42.459033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:42.505178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 744/1489] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:42.792120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4bcc-a91d-4f65-84c0-72c66715e37f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:42.794550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d7ec4bcc-a91d-4f65-84c0-72c66715e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:42.843552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:42.915289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4bcc-a91d-4f65-84c0-72c66715e37f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:42.993357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa45d9-844c-4f4d-b1b4-48698dbeda6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:42.993965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa45d9-844c-4f4d-b1b4-48698dbeda6a service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:42.994236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa45d9-844c-4f4d-b1b4-48698dbeda6a service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:42.994508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa45d9-844c-4f4d-b1b4-48698dbeda6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:42.996459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-45aa45d9-844c-4f4d-b1b4-48698dbeda6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:43.084210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:43.117627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa45d9-844c-4f4d-b1b4-48698dbeda6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:43.119069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa45d9-844c-4f4d-b1b4-48698dbeda6a service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:43.138113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 746/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:45.501560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d74f01-0bd8-4381-8c74-6c3c8a59e9a9 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.502143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d74f01-0bd8-4381-8c74-6c3c8a59e9a9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.502143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d74f01-0bd8-4381-8c74-6c3c8a59e9a9 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.504362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 745/1491] 10.222.0.44 () {54 vars in 927 bytes} [Tue Oct 4 08:46:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:45.513145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602fd7-f9a2-4972-a988-e17c0e4988d8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.513569 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602fd7-f9a2-4972-a988-e17c0e4988d8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.513780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602fd7-f9a2-4972-a988-e17c0e4988d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.612799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-da602fd7-f9a2-4972-a988-e17c0e4988d8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:45.614250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-da602fd7-f9a2-4972-a988-e17c0e4988d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:45.716133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 747/1492] 10.222.0.44 () {58 vars in 1024 bytes} [Tue Oct 4 08:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:46.014420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d74f01-0bd8-4381-8c74-6c3c8a59e9a9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:46.016061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-10d74f01-0bd8-4381-8c74-6c3c8a59e9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:46.133819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d74f01-0bd8-4381-8c74-6c3c8a59e9a9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:46.220630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58580487-0b41-418c-8fa1-ad3736156ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:46.221569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58580487-0b41-418c-8fa1-ad3736156ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:46.221908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58580487-0b41-418c-8fa1-ad3736156ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:46.222264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58580487-0b41-418c-8fa1-ad3736156ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:46.224869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-58580487-0b41-418c-8fa1-ad3736156ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:46.237922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:46.346650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58580487-0b41-418c-8fa1-ad3736156ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:46.348113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58580487-0b41-418c-8fa1-ad3736156ea0 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:46.366250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 746/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:55.714347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=82 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:56.363913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=50 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:59.038154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.038706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.038706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.118646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:59.120017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.221588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 748/1494] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:59.229986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45b12a-0f35-4153-b8ef-824fad5db54a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.230526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45b12a-0f35-4153-b8ef-824fad5db54a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.233727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45b12a-0f35-4153-b8ef-824fad5db54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.323371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4a45b12a-0f35-4153-b8ef-824fad5db54a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:59.324767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4a45b12a-0f35-4153-b8ef-824fad5db54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.431154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 747/1495] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:59.436365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.437584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.449594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2e530-2ccf-4365-950c-9fb3ce227666 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.520998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:59.558267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.558913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.559187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.559455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.560120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:59.560417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.561425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:47:59.561425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:47:59.561425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:47:59.562302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.573664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 749/1496] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:47:59.582163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45b12a-0f35-4153-b8ef-824fad5db54a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.591823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45b12a-0f35-4153-b8ef-824fad5db54a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.665613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:59.701221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.701839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.702115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.702388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.703016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.704842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING py.warnings [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:47:59.704842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: warnings.warn(msg) Oct 04 08:47:59.704842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: Oct 04 08:47:59.706038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.768521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 748/1497] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:59.772931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.774394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.808207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2976307-f3be-418c-a12a-03ab0d3aa112 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.907607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.908239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.908510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.908776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.909406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.910403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:47:59.910403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:47:59.910403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:47:59.911596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.957054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 750/1498] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:59.961416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.962948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.029940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870e307-1fe9-47eb-a20e-bae4f7db9616 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:00.113903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:00.126374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:00.126993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.127277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.127550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.128305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:00.129608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING py.warnings [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:48:00.129608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: warnings.warn(msg) Oct 04 08:48:00.129608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: Oct 04 08:48:00.130519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:00.143033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 749/1499] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:00.147244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:00.148773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.160160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4115c-05c5-4138-a4dd-901468bf227c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:00.186086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:00.260879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c8253-1f74-4a0e-bea9-324d07a88e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:00.261476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8253-1f74-4a0e-bea9-324d07a88e01 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.261786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8253-1f74-4a0e-bea9-324d07a88e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.262049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8253-1f74-4a0e-bea9-324d07a88e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.262697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c8253-1f74-4a0e-bea9-324d07a88e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:00.263684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-1f8c8253-1f74-4a0e-bea9-324d07a88e01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:48:00.263684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:48:00.263684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:48:00.264585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c8253-1f74-4a0e-bea9-324d07a88e01 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:00.277059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 751/1500] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:00.283122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:00.284662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.320396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabcb06-9ddd-4759-8b96-5fd659d4104f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:00.418488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd78b6-d3aa-4802-869a-f9218a8f44ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:00.419149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd78b6-d3aa-4802-869a-f9218a8f44ff admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.419415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd78b6-d3aa-4802-869a-f9218a8f44ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.419680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd78b6-d3aa-4802-869a-f9218a8f44ff admin admin] PATH_INFO: `/v3/projects/13f8c5ea5c3948ca8587001b184b87f3/users/60b13c012acc4cf38f9a32d35813c06c/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.441933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd78b6-d3aa-4802-869a-f9218a8f44ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f8c5ea5c3948ca8587001b184b87f3, user_id=60b13c012acc4cf38f9a32d35813c06c, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:00.442963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING py.warnings [None req-61cd78b6-d3aa-4802-869a-f9218a8f44ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:48:00.442963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: warnings.warn(msg) Oct 04 08:48:00.442963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: Oct 04 08:48:00.444921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd78b6-d3aa-4802-869a-f9218a8f44ff admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:00.470454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 750/1501] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:48:00 2022] PUT /identity/v3/projects/13f8c5ea5c3948ca8587001b184b87f3/users/60b13c012acc4cf38f9a32d35813c06c/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:00.482808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6ba1-acf1-465a-a7a5-9389902a59be None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.483066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6ba1-acf1-465a-a7a5-9389902a59be None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.483363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde6ba1-acf1-465a-a7a5-9389902a59be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.542283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-bfde6ba1-acf1-465a-a7a5-9389902a59be None None] MFA Rules not processed for user `60b13c012acc4cf38f9a32d35813c06c`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:00.543442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bfde6ba1-acf1-465a-a7a5-9389902a59be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.640021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 752/1502] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:00.660589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fb086-e683-48af-983f-4ca6819e375f None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.660946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fb086-e683-48af-983f-4ca6819e375f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.661321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fb086-e683-48af-983f-4ca6819e375f None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.663321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 751/1503] 10.222.0.44 () {54 vars in 963 bytes} [Tue Oct 4 08:48:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:48:00.669993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a9e0e-c41c-4f3d-a1bb-c226b965698e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.670281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a9e0e-c41c-4f3d-a1bb-c226b965698e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.670551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a9e0e-c41c-4f3d-a1bb-c226b965698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.703672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:00.755943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-406a9e0e-c41c-4f3d-a1bb-c226b965698e None None] MFA Rules not processed for user `d877daa4c2c14d0c8d281aeebab27ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:00.757776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-406a9e0e-c41c-4f3d-a1bb-c226b965698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.857288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 753/1504] 10.222.0.44 () {58 vars in 1060 bytes} [Tue Oct 4 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:00.865164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fb086-e683-48af-983f-4ca6819e375f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:00.867083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-641fb086-e683-48af-983f-4ca6819e375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.922889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:00.990847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fb086-e683-48af-983f-4ca6819e375f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:01.066707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:01.067303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:01.067572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:01.067808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:01.069467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:01.176182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:01.177737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:01.181136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:01.201638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 752/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:01.259407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68e0fe-1439-4ecd-b179-45bcaa607455 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:01.259864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68e0fe-1439-4ecd-b179-45bcaa607455 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:01.259997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68e0fe-1439-4ecd-b179-45bcaa607455 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:01.261329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 754/1506] 10.222.0.44 () {52 vars in 830 bytes} [Tue Oct 4 08:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:48:01.265175 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:01.266520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:01.384886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75e79b-5c60-434a-b0ab-13db4f9d6307 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:01.482921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:01.483580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:01.483855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:01.484154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] PATH_INFO: `/v3/users/60b13c012acc4cf38f9a32d35813c06c` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:01.499244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60b13c012acc4cf38f9a32d35813c06c)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:01.500306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING py.warnings [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:48:01.500306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: warnings.warn(msg) Oct 04 08:48:01.500306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: Oct 04 08:48:01.501561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:01.513922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:01.576691 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60b13c012acc4cf38f9a32d35813c06c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.577669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60b13c012acc4cf38f9a32d35813c06c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.597740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60b13c012acc4cf38f9a32d35813c06c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.598365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60b13c012acc4cf38f9a32d35813c06c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.624747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60b13c012acc4cf38f9a32d35813c06c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.641559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 753/1507] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:48:01 2022] DELETE /identity/v3/users/60b13c012acc4cf38f9a32d35813c06c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:01.645993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:01.646303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:01.646580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:01.707410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:01.708686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:01.789526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:01.802958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 755/1508] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:01.813052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:01.814305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:01.933778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf0659-a824-4598-93ce-57286c9e4038 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:01.992734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:02.013280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0107f-30c9-41e4-8f8e-abdfef4c3aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.013894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0107f-30c9-41e4-8f8e-abdfef4c3aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.014168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0107f-30c9-41e4-8f8e-abdfef4c3aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.014433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0107f-30c9-41e4-8f8e-abdfef4c3aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.016062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1ac0107f-30c9-41e4-8f8e-abdfef4c3aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:02.127599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0107f-30c9-41e4-8f8e-abdfef4c3aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.129094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0107f-30c9-41e4-8f8e-abdfef4c3aa1 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.148443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 754/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:02.475327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.476648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:02.488048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2cc6e-d78b-47cb-ad95-62bdc2d9207f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.587585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.588241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.588489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.588767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] PATH_INFO: `/v3/projects/13f8c5ea5c3948ca8587001b184b87f3` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.599054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f8c5ea5c3948ca8587001b184b87f3)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.600096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:48:02.600096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:48:02.600096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:48:02.601301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.678108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f8c5ea5c3948ca8587001b184b87f3', 'request_id': 'req-4567a198-de90-4c95-b6ad-162933dd8b08'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:02.694904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f8c5ea5c3948ca8587001b184b87f3', 'request_id': 'req-4567a198-de90-4c95-b6ad-162933dd8b08'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:02.701913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f8c5ea5c3948ca8587001b184b87f3', 'request_id': 'req-4567a198-de90-4c95-b6ad-162933dd8b08'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:02.729472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] The token cache is being invalidate because project 13f8c5ea5c3948ca8587001b184b87f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:48:02.729736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:02.730200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4567a198-de90-4c95-b6ad-162933dd8b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:02.731623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 756/1510] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:48:02 2022] DELETE /identity/v3/projects/13f8c5ea5c3948ca8587001b184b87f3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:12.145750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=62 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:12.727324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:33.161013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003f2a-36f8-4f8c-93c9-709221808161 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:33.161984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003f2a-36f8-4f8c-93c9-709221808161 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:33.161984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003f2a-36f8-4f8c-93c9-709221808161 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:33.163712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:33.169743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b303-f239-457d-8eff-4408ce2617f2 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:33.170487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b303-f239-457d-8eff-4408ce2617f2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:33.170644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b303-f239-457d-8eff-4408ce2617f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:33.260490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ac52b303-f239-457d-8eff-4408ce2617f2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:33.261619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ac52b303-f239-457d-8eff-4408ce2617f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:33.372551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:33.526144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003f2a-36f8-4f8c-93c9-709221808161 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:33.530201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-60003f2a-36f8-4f8c-93c9-709221808161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:33.662591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003f2a-36f8-4f8c-93c9-709221808161 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:33.669753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80089357-d752-462f-ba63-ddaa6f316ef0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:33.670365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089357-d752-462f-ba63-ddaa6f316ef0 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:33.670647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089357-d752-462f-ba63-ddaa6f316ef0 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:33.670919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089357-d752-462f-ba63-ddaa6f316ef0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:33.680604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80089357-d752-462f-ba63-ddaa6f316ef0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:33.682318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80089357-d752-462f-ba63-ddaa6f316ef0 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:33.692246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 756/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:34.784196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b07604-65a7-41cb-9774-95c6e3c5ddce None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:34.784780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b07604-65a7-41cb-9774-95c6e3c5ddce None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:34.784780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b07604-65a7-41cb-9774-95c6e3c5ddce None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:34.786908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:34.793403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b77bc-fc24-43dc-b59e-898da36afd29 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:34.793900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b77bc-fc24-43dc-b59e-898da36afd29 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:34.793900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b77bc-fc24-43dc-b59e-898da36afd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:34.814898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-015b77bc-fc24-43dc-b59e-898da36afd29 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:34.816097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-015b77bc-fc24-43dc-b59e-898da36afd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:34.824460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:34.981143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982a8a2-47b0-4767-8bfc-048964ec6f7d None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:34.981452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982a8a2-47b0-4767-8bfc-048964ec6f7d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:34.981721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982a8a2-47b0-4767-8bfc-048964ec6f7d None None] PATH_INFO: `/v3` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:34.983024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 759/1516] 10.222.0.52 () {54 vars in 925 bytes} [Tue Oct 4 08:49:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:49:34.990210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775436b-fffe-47ae-9f40-1a55d98da321 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:34.990483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775436b-fffe-47ae-9f40-1a55d98da321 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:34.990762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775436b-fffe-47ae-9f40-1a55d98da321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:34.995409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982a8a2-47b0-4767-8bfc-048964ec6f7d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:35.011039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982a8a2-47b0-4767-8bfc-048964ec6f7d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:35.016912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462d0bb-954d-4b9f-9d56-4c35dd6e966a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:35.017534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462d0bb-954d-4b9f-9d56-4c35dd6e966a service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:35.017809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462d0bb-954d-4b9f-9d56-4c35dd6e966a service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:35.018064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462d0bb-954d-4b9f-9d56-4c35dd6e966a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:35.026920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462d0bb-954d-4b9f-9d56-4c35dd6e966a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:35.028524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462d0bb-954d-4b9f-9d56-4c35dd6e966a service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:35.038017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 760/1517] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:35.055906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d775436b-fffe-47ae-9f40-1a55d98da321 None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:35.057151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d775436b-fffe-47ae-9f40-1a55d98da321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:35.116631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:35.137673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 758/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Oct 4 08:49:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:44.213786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c3960-808b-4112-9593-ff90b777b29e None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:44.214258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c3960-808b-4112-9593-ff90b777b29e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:44.214363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c3960-808b-4112-9593-ff90b777b29e None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:44.215663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 761/1519] 10.222.0.44 () {54 vars in 961 bytes} [Tue Oct 4 08:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:44.220752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824b0eb-a10c-4495-8133-6bf8f3fce958 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:44.221180 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824b0eb-a10c-4495-8133-6bf8f3fce958 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:44.221305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824b0eb-a10c-4495-8133-6bf8f3fce958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:44.242468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c824b0eb-a10c-4495-8133-6bf8f3fce958 None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:44.243589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c824b0eb-a10c-4495-8133-6bf8f3fce958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:44.253204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 759/1520] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Oct 4 08:49:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:44.260749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c3960-808b-4112-9593-ff90b777b29e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:44.274360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c3960-808b-4112-9593-ff90b777b29e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:44.279766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28530b52-cac5-469f-a485-cbda61425fe0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:44.280384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530b52-cac5-469f-a485-cbda61425fe0 service placement] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:44.280660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530b52-cac5-469f-a485-cbda61425fe0 service placement] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:44.281013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28530b52-cac5-469f-a485-cbda61425fe0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:44.289260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28530b52-cac5-469f-a485-cbda61425fe0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:44.290805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28530b52-cac5-469f-a485-cbda61425fe0 service placement] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:44.300278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 762/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:46.523093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f4b5a-95f5-403b-b1fc-658f965351ca None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:46.523930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f4b5a-95f5-403b-b1fc-658f965351ca None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:46.523930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f4b5a-95f5-403b-b1fc-658f965351ca None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:46.525757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:49:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:46.533216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc814-7793-4dba-afb4-85ed9d350545 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:46.533675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc814-7793-4dba-afb4-85ed9d350545 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:46.533814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc814-7793-4dba-afb4-85ed9d350545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:46.561877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2b8cc814-7793-4dba-afb4-85ed9d350545 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:46.563632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2b8cc814-7793-4dba-afb4-85ed9d350545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:46.576899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:46.739528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f4b5a-95f5-403b-b1fc-658f965351ca None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:46.762476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f4b5a-95f5-403b-b1fc-658f965351ca None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:46.768173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef7220-e27c-4a1c-aa57-2d8693915d9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:46.768774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef7220-e27c-4a1c-aa57-2d8693915d9d service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:46.769053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef7220-e27c-4a1c-aa57-2d8693915d9d service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:46.769323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef7220-e27c-4a1c-aa57-2d8693915d9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:46.777695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef7220-e27c-4a1c-aa57-2d8693915d9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:46.779284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef7220-e27c-4a1c-aa57-2d8693915d9d service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:46.792659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 761/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:47.851457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853238f-b2fb-4589-8f45-63c070d5cc72 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:47.852051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853238f-b2fb-4589-8f45-63c070d5cc72 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:47.852051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853238f-b2fb-4589-8f45-63c070d5cc72 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:47.853424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:47.860579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e82d11-5d6a-4fbd-9666-7c3529a58c66 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:47.861019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e82d11-5d6a-4fbd-9666-7c3529a58c66 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:47.861137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e82d11-5d6a-4fbd-9666-7c3529a58c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:47.891074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b4e82d11-5d6a-4fbd-9666-7c3529a58c66 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:47.892311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b4e82d11-5d6a-4fbd-9666-7c3529a58c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:47.901230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:48.056390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853238f-b2fb-4589-8f45-63c070d5cc72 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:48.074097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853238f-b2fb-4589-8f45-63c070d5cc72 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:48.080201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65809831-cf7a-4300-9a42-b1e9ce4d3467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:48.080795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65809831-cf7a-4300-9a42-b1e9ce4d3467 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:48.081074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65809831-cf7a-4300-9a42-b1e9ce4d3467 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:48.081365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65809831-cf7a-4300-9a42-b1e9ce4d3467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:48.089945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65809831-cf7a-4300-9a42-b1e9ce4d3467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:48.091523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65809831-cf7a-4300-9a42-b1e9ce4d3467 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:48.101021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 765/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:52.110930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6390b1-42a5-4b61-8228-a070d7639b1c None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:52.111507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6390b1-42a5-4b61-8228-a070d7639b1c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:52.111507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6390b1-42a5-4b61-8228-a070d7639b1c None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:52.112825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:52.118617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ee65-452e-4a42-b882-ee0a56de57de None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:52.119049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ee65-452e-4a42-b882-ee0a56de57de None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:52.119162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ee65-452e-4a42-b882-ee0a56de57de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:52.138954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-8981ee65-452e-4a42-b882-ee0a56de57de None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:52.140166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8981ee65-452e-4a42-b882-ee0a56de57de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:52.147851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:52.429818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6390b1-42a5-4b61-8228-a070d7639b1c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:52.431974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3b6390b1-42a5-4b61-8228-a070d7639b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:52.517106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6390b1-42a5-4b61-8228-a070d7639b1c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:52.521037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8d355-ab68-4e66-8611-0395ae0089bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:52.521646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d355-ab68-4e66-8611-0395ae0089bd service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:52.521922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d355-ab68-4e66-8611-0395ae0089bd service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:52.522191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8d355-ab68-4e66-8611-0395ae0089bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:52.531694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8d355-ab68-4e66-8611-0395ae0089bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:52.533153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8d355-ab68-4e66-8611-0395ae0089bd service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:52.542786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 764/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:54.646253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc75c4-8269-499c-9534-f7180e8e50ab None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:54.646956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc75c4-8269-499c-9534-f7180e8e50ab None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:54.646956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc75c4-8269-499c-9534-f7180e8e50ab None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:54.648196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:49:54.653559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99b71d-a6fd-4e5b-a47b-aceb845384e3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:54.653977 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99b71d-a6fd-4e5b-a47b-aceb845384e3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:54.654096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99b71d-a6fd-4e5b-a47b-aceb845384e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:54.673964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ed99b71d-a6fd-4e5b-a47b-aceb845384e3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:54.675146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ed99b71d-a6fd-4e5b-a47b-aceb845384e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:54.685306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:54.986988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc75c4-8269-499c-9534-f7180e8e50ab None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:55.001159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc75c4-8269-499c-9534-f7180e8e50ab None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:55.005840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a5bac-c1ed-44eb-bd45-8e206faf6e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:55.006434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5bac-c1ed-44eb-bd45-8e206faf6e4e service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:55.006706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5bac-c1ed-44eb-bd45-8e206faf6e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:55.006955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a5bac-c1ed-44eb-bd45-8e206faf6e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:55.015996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a5bac-c1ed-44eb-bd45-8e206faf6e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:55.017410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a5bac-c1ed-44eb-bd45-8e206faf6e4e service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:55.027504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 768/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:58.676741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c032a-5289-40d0-ab1c-a4f2a516e370 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:58.677295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c032a-5289-40d0-ab1c-a4f2a516e370 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:58.677295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c032a-5289-40d0-ab1c-a4f2a516e370 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:58.678548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:58.684914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aa0da-cd1c-47ad-b04c-f0e7b9fa0c3a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:58.685358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aa0da-cd1c-47ad-b04c-f0e7b9fa0c3a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:58.685475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aa0da-cd1c-47ad-b04c-f0e7b9fa0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:58.697531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:58.711773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-da0aa0da-cd1c-47ad-b04c-f0e7b9fa0c3a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:58.713047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-da0aa0da-cd1c-47ad-b04c-f0e7b9fa0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:58.722017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:59.048012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c032a-5289-40d0-ab1c-a4f2a516e370 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:59.050190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-be4c032a-5289-40d0-ab1c-a4f2a516e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:59.142351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c032a-5289-40d0-ab1c-a4f2a516e370 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:59.146305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71fd5-93f8-4e2d-b0b2-d13381da29c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:59.146909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71fd5-93f8-4e2d-b0b2-d13381da29c0 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:59.147186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71fd5-93f8-4e2d-b0b2-d13381da29c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:59.147452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71fd5-93f8-4e2d-b0b2-d13381da29c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:59.156597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd71fd5-93f8-4e2d-b0b2-d13381da29c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:59.158023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd71fd5-93f8-4e2d-b0b2-d13381da29c0 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:59.167532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 767/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:01.287906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18095906-b44b-480d-a783-c902e7dd9997 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:01.288495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18095906-b44b-480d-a783-c902e7dd9997 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:01.288495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18095906-b44b-480d-a783-c902e7dd9997 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:01.289810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:50:01.295996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0e5c-5577-47bb-85bf-d635fbf8a00e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:01.296670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0e5c-5577-47bb-85bf-d635fbf8a00e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:01.296769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd0e5c-5577-47bb-85bf-d635fbf8a00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:01.328440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e4cd0e5c-5577-47bb-85bf-d635fbf8a00e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:01.329669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e4cd0e5c-5577-47bb-85bf-d635fbf8a00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:01.344707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:01.649267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18095906-b44b-480d-a783-c902e7dd9997 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:01.651399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-18095906-b44b-480d-a783-c902e7dd9997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:01.670059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18095906-b44b-480d-a783-c902e7dd9997 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:01.675134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a501e8-37d5-417d-ac07-26f1714aec83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:01.676034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a501e8-37d5-417d-ac07-26f1714aec83 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:01.676441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a501e8-37d5-417d-ac07-26f1714aec83 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:01.676836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a501e8-37d5-417d-ac07-26f1714aec83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:01.691470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a501e8-37d5-417d-ac07-26f1714aec83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:01.693731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a501e8-37d5-417d-ac07-26f1714aec83 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:01.711178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 771/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:11.320253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:11.703794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=8 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:28.783401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fe116-306f-4433-a657-a8e6ba0f6e95 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.784077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fe116-306f-4433-a657-a8e6ba0f6e95 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.784077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fe116-306f-4433-a657-a8e6ba0f6e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.814232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-aa3fe116-306f-4433-a657-a8e6ba0f6e95 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:28.815467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-aa3fe116-306f-4433-a657-a8e6ba0f6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:28.826573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 769/1540] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:28.835408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab9266-811e-4a25-a05f-9f6f845dd561 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.835997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab9266-811e-4a25-a05f-9f6f845dd561 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.835997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab9266-811e-4a25-a05f-9f6f845dd561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.852244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad775-4a45-4d2b-a8c6-007dd9758351 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.852640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad775-4a45-4d2b-a8c6-007dd9758351 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.852874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad775-4a45-4d2b-a8c6-007dd9758351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.871264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-88ab9266-811e-4a25-a05f-9f6f845dd561 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:28.872528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-88ab9266-811e-4a25-a05f-9f6f845dd561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:28.889559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-439ad775-4a45-4d2b-a8c6-007dd9758351 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:28.890976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-439ad775-4a45-4d2b-a8c6-007dd9758351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:28.896234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 772/1541] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:28.897545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbb52b-c45c-47a0-bbd1-017a675cf4ef None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.897832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbb52b-c45c-47a0-bbd1-017a675cf4ef None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.898110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbb52b-c45c-47a0-bbd1-017a675cf4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.913668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 770/1542] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:51:28.916806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55908-fd10-4944-891e-9fdb7c115cdc None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.917094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55908-fd10-4944-891e-9fdb7c115cdc None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.917373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55908-fd10-4944-891e-9fdb7c115cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.938708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-73f55908-fd10-4944-891e-9fdb7c115cdc None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:28.939990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-73f55908-fd10-4944-891e-9fdb7c115cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:28.954677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 771/1543] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:28.954911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1ffbb52b-c45c-47a0-bbd1-017a675cf4ef None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:28.955517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55908-fd10-4944-891e-9fdb7c115cdc None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:28.956180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1ffbb52b-c45c-47a0-bbd1-017a675cf4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:28.968423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55908-fd10-4944-891e-9fdb7c115cdc None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:28.974963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:28.975584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.976025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.977810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 773/1544] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:28.978081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.979034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:28.979354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:28.980427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING py.warnings [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:51:28.980427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: warnings.warn(msg) Oct 04 08:51:28.980427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: Oct 04 08:51:28.980736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36366980-e25d-41d5-a93c-bfe5aee288b7 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.980736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36366980-e25d-41d5-a93c-bfe5aee288b7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.980983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36366980-e25d-41d5-a93c-bfe5aee288b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.981321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afcb22-7bc5-4dec-b834-8a57a57a483e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:28.995636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 772/1545] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:28.996864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533824e8-905c-4349-bbfd-1e64004c0273 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.997148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533824e8-905c-4349-bbfd-1e64004c0273 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.997426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533824e8-905c-4349-bbfd-1e64004c0273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.006627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-36366980-e25d-41d5-a93c-bfe5aee288b7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.008474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-36366980-e25d-41d5-a93c-bfe5aee288b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.017496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 774/1546] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.019159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011fafa-19be-4a5d-ba17-984285ea4814 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.019285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-533824e8-905c-4349-bbfd-1e64004c0273 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.019762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011fafa-19be-4a5d-ba17-984285ea4814 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.020343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011fafa-19be-4a5d-ba17-984285ea4814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.020482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-533824e8-905c-4349-bbfd-1e64004c0273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.029021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 773/1547] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.031249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1229b4-d40a-4378-8a14-8d007257e082 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.031249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1229b4-d40a-4378-8a14-8d007257e082 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.031249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1229b4-d40a-4378-8a14-8d007257e082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.043678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c011fafa-19be-4a5d-ba17-984285ea4814 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.045301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c011fafa-19be-4a5d-ba17-984285ea4814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.052347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ed1229b4-d40a-4378-8a14-8d007257e082 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.053550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ed1229b4-d40a-4378-8a14-8d007257e082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.054861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 775/1548] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.056454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42c7f2-bac1-4506-9ba2-cf7d09e25103 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.056913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42c7f2-bac1-4506-9ba2-cf7d09e25103 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.057349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42c7f2-bac1-4506-9ba2-cf7d09e25103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.062203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 774/1549] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.062971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1229b4-d40a-4378-8a14-8d007257e082 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.073456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1229b4-d40a-4378-8a14-8d007257e082 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.078487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e05ab1-c080-43bf-a24e-d2613fe3ae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.079101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e05ab1-c080-43bf-a24e-d2613fe3ae66 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.079380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e05ab1-c080-43bf-a24e-d2613fe3ae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.080034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0a42c7f2-bac1-4506-9ba2-cf7d09e25103 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.080171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e05ab1-c080-43bf-a24e-d2613fe3ae66 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.080745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e05ab1-c080-43bf-a24e-d2613fe3ae66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.081504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0a42c7f2-bac1-4506-9ba2-cf7d09e25103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.082661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e05ab1-c080-43bf-a24e-d2613fe3ae66 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.090028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 776/1550] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.091309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42c7f2-bac1-4506-9ba2-cf7d09e25103 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.102001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42c7f2-bac1-4506-9ba2-cf7d09e25103 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.107019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5d551-e0a9-4e8c-be53-219efa042758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.107871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5d551-e0a9-4e8c-be53-219efa042758 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.108417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5d551-e0a9-4e8c-be53-219efa042758 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.108694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5d551-e0a9-4e8c-be53-219efa042758 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.109476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5d551-e0a9-4e8c-be53-219efa042758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:29.109785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5d551-e0a9-4e8c-be53-219efa042758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.111439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5d551-e0a9-4e8c-be53-219efa042758 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.124428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 777/1551] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:29.125916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb565-4d39-4667-9f5b-79abf0f84b93 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.126378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb565-4d39-4667-9f5b-79abf0f84b93 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.126647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6deb565-4d39-4667-9f5b-79abf0f84b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.134733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 775/1552] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.136026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1289-fb65-4405-907f-716679c58310 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.136315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1289-fb65-4405-907f-716679c58310 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.136590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1289-fb65-4405-907f-716679c58310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.150307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c6deb565-4d39-4667-9f5b-79abf0f84b93 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.151477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c6deb565-4d39-4667-9f5b-79abf0f84b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.159699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-58eb1289-fb65-4405-907f-716679c58310 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.160960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-58eb1289-fb65-4405-907f-716679c58310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.217097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 778/1553] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.218128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb565-4d39-4667-9f5b-79abf0f84b93 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.226621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 776/1554] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.227343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6deb565-4d39-4667-9f5b-79abf0f84b93 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.227911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c23b-1065-4aa0-b9cf-ac60d58183ad None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.228209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c23b-1065-4aa0-b9cf-ac60d58183ad None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.228476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3c23b-1065-4aa0-b9cf-ac60d58183ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.236907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.237850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.238121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.238383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.239006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:29.239303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.240935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.250299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d2c3c23b-1065-4aa0-b9cf-ac60d58183ad None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.251503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d2c3c23b-1065-4aa0-b9cf-ac60d58183ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.252665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 779/1555] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:29.253432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.260570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 777/1556] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.261766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b15c-f2b5-41f1-9c17-a732e073f02e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.262045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b15c-f2b5-41f1-9c17-a732e073f02e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.262301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b15c-f2b5-41f1-9c17-a732e073f02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.267684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246e01-978c-4755-b535-d20c1d880531 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.272263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.272862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.273276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.273543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.274180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:29.274542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.276155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.282136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6900b15c-f2b5-41f1-9c17-a732e073f02e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.283336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6900b15c-f2b5-41f1-9c17-a732e073f02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.288818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 780/1557] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:29.289587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.298121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 778/1558] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.298121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900b15c-f2b5-41f1-9c17-a732e073f02e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.299700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba3c60-5063-4b50-9195-dcbcd2bb0fa4 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.304308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.304900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.305179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.305452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.306143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.307161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:51:29.307161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:51:29.307161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:51:29.307845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900b15c-f2b5-41f1-9c17-a732e073f02e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.308374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.312402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc4410-7e3e-4215-86b0-32601c650aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.312997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc4410-7e3e-4215-86b0-32601c650aaa admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.313276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc4410-7e3e-4215-86b0-32601c650aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.313784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc4410-7e3e-4215-86b0-32601c650aaa admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.314185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc4410-7e3e-4215-86b0-32601c650aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.315201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING py.warnings [None req-b9cc4410-7e3e-4215-86b0-32601c650aaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:51:29.315201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: warnings.warn(msg) Oct 04 08:51:29.315201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: Oct 04 08:51:29.316538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc4410-7e3e-4215-86b0-32601c650aaa admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.336598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 781/1559] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.337609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.340093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 779/1560] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.341509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467886-0d7a-479f-a52d-189190cc65b1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.341825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467886-0d7a-479f-a52d-189190cc65b1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.342093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467886-0d7a-479f-a52d-189190cc65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.360883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c3467886-0d7a-479f-a52d-189190cc65b1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.362185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c3467886-0d7a-479f-a52d-189190cc65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.399073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae6f44-164f-4359-8668-5b3393e6e0c2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.403717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.404354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.404627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.404895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.405533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:29.405839 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.407415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.411372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:29.415936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:29.419052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 782/1561] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:29.420570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.430765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f1b75-071a-4994-ba95-70be206686fb admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.432536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 780/1562] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.433560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3467886-0d7a-479f-a52d-189190cc65b1 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.444197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3467886-0d7a-479f-a52d-189190cc65b1 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.444624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0eefd-a93c-43df-8b64-f48f5a515bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.445429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0eefd-a93c-43df-8b64-f48f5a515bcb admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.445706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0eefd-a93c-43df-8b64-f48f5a515bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.446010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0eefd-a93c-43df-8b64-f48f5a515bcb admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.446722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0eefd-a93c-43df-8b64-f48f5a515bcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.448741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0eefd-a93c-43df-8b64-f48f5a515bcb admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.449245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.449852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.450132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.450527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.451186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:29.451492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.453120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.464969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 781/1563] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:29.472188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.476006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092af0a-c172-49cd-b741-9ecbef75405a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.480857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.481503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.481778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.482061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.482664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.484587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.487561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 783/1564] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.488868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcb074-4892-4455-9b21-95f5e3ee5508 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.489282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcb074-4892-4455-9b21-95f5e3ee5508 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.489560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcb074-4892-4455-9b21-95f5e3ee5508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.511032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-87bcb074-4892-4455-9b21-95f5e3ee5508 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:29.512427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-87bcb074-4892-4455-9b21-95f5e3ee5508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:29.515743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 782/1565] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.516588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.573652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 784/1566] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:29.574418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcb074-4892-4455-9b21-95f5e3ee5508 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.579667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05760da-1d32-4f40-bcea-bdfda3f91896 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.583711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcb074-4892-4455-9b21-95f5e3ee5508 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.584310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.584928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.585211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.585486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.586123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.588535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.589665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.590256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.590556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.590831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.591465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.593064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.604551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 785/1567] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:29.605358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.612877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 783/1568] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.616508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.617652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca217a-6a3a-4109-b89b-fbf5ce4dbc0a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.622466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.623061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.623338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.623620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.624258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.626131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.626331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e49a49-6018-487c-a0a7-ef007a93b95a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.630944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.631546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.631831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.632135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.632777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:29.633083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.634717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.648246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 784/1569] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:29.656497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.656936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 786/1570] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.657744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.666717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517713f-1bed-48d5-85dd-4e7e2fd6c85b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.721789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c592f4-c33c-4b03-9341-84d77bcfeae0 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.723426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.724083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.724365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.724636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.725275 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.726630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.727178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.727296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.727559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.727834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.728495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.730382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.756713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 787/1571] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.756713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.757004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 785/1572] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.757004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.768817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e4f09-a9fe-48b7-8c2f-a5a284bbf7e6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.779084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:29.804053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:29.823614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c8209-61ff-41b2-934e-72098463262c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.828256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.828410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.828817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.829025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.829139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.829305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.829402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.829576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.829996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:29.830210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.830316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.831930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.832140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.843952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 786/1573] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:29.844997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.855418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd44a3-9699-4167-ab4f-86b603963cd3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.857259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 788/1574] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.858155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.860314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.860987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.861265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.861532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.862167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.864308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.868960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4913c63-a22f-4e47-bee6-d4cf069e37fe admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.873605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.874203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.874483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.874753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.875392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.876119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 787/1575] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:29.876638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.880115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.886239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a421032-e564-4623-87cd-28864bfe1047 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.893432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 789/1576] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:29.893589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.893705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.893798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.893798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.893798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.893798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.895052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.900494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883284f-c468-420b-ab93-f6ab6f7224e8 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.905061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.905661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.905931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.906204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.906826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.908748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:29.922680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 788/1577] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.923544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.936304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 790/1578] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:29.937143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:29.947514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d28cc-a0cd-44e2-95c7-cd3ef48cce7a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.988273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce0f0c-37de-4598-a004-7e620aeacf40 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:29.997383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:29.997383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:29.997383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:29.997383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:29.997796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:29.997796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.006654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 789/1579] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.007500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.010415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.011050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.011326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.011596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.012270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.014179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.016911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351dda9-0c52-4ac2-a024-63f61486710a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.021829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.022535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.022909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.023281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.023944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.026265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.038032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 791/1580] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:29 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:30.041303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.049342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a99dc-cdb9-44d3-a8ba-a38151d2153e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.054001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.054670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.055019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.055360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.058909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 790/1581] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:30 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:30.059065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.059176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.059268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.072051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 792/1582] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.073410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.083220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bb8d4-ae1a-4b8f-a6ad-c02b4806defe admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.116835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:30.122277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc750-2cff-4038-a25d-2ad02371fc29 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.128240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.129012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.129292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.129579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] PATH_INFO: `/v3/projects/0c4d8cf4a0b74162b6f73a0b2d88ff09/users/91ccd41938954a36adb45c70e6ff41de/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.142971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.143703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.143830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4d8cf4a0b74162b6f73a0b2d88ff09, user_id=91ccd41938954a36adb45c70e6ff41de, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.144145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.144484 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.145221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.146536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.146957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.157878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 793/1583] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.158732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.164141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 791/1584] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:30 2022] PUT /identity/v3/projects/0c4d8cf4a0b74162b6f73a0b2d88ff09/users/91ccd41938954a36adb45c70e6ff41de/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:30.164141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.169097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b5a94-5490-4be2-8ebd-e4e89795fd6a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.177720 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:30.223358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8a822-0d8c-4f76-a19e-710142656ed8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.226991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.227718 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.227853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.228054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.228305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.228499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.228776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.228942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.229072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.229672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.230857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.231253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.241500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 792/1585] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.242364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.252223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62176d-b77b-45cd-8d5a-79a7dc4b3386 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.255817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 794/1586] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:30 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:30.256557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.257457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.258083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.258408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.258715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.259348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.260990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.266211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f880992-2f29-43b4-9172-1fb352432650 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.270835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.271424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.271545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 793/1587] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.271699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.271998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.272260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.272665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.274229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.283803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f6739-a8c8-4ebb-950e-51236bc9db53 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.285298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 795/1588] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.286108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.290690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac0779-deef-4637-bdfb-5d0b67dceb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.291262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0779-deef-4637-bdfb-5d0b67dceb08 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.291569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0779-deef-4637-bdfb-5d0b67dceb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.291823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0779-deef-4637-bdfb-5d0b67dceb08 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.292471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac0779-deef-4637-bdfb-5d0b67dceb08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.294376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac0779-deef-4637-bdfb-5d0b67dceb08 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.296157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d08f7b-2b67-4bca-817e-c90e694c95c9 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.300654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.301235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.301507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.301775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.302400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.303965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.317810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 796/1589] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.317810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.318099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 794/1590] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:30 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:30.318099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade41334-9744-41af-aade-221968863421 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.318099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade41334-9744-41af-aade-221968863421 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.318099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade41334-9744-41af-aade-221968863421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.327671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0121640-5ab2-4da4-ba98-ab288a72ffd6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.332585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.333286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.333510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.333787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] PATH_INFO: `/v3/projects/269445aa5f0745a4ac2eff02524fbcc0/users/a2b91a1c4acc4cc88cd98a3215ebcc98/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.336146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ade41334-9744-41af-aade-221968863421 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:30.337371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ade41334-9744-41af-aade-221968863421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:30.344942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 795/1591] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:30.345522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade41334-9744-41af-aade-221968863421 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.352597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=269445aa5f0745a4ac2eff02524fbcc0, user_id=a2b91a1c4acc4cc88cd98a3215ebcc98, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.353790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:51:30.353790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:51:30.353790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:51:30.355776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.362761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade41334-9744-41af-aade-221968863421 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.368115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.368930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.369204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.369473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.370115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.371740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.373622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 797/1592] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:30 2022] PUT /identity/v3/projects/269445aa5f0745a4ac2eff02524fbcc0/users/a2b91a1c4acc4cc88cd98a3215ebcc98/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:30.374521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.382316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 796/1593] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.385751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.395638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84628-eb6a-4a76-9b71-5fca44ba8eaa admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.441961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b56e-53a3-4605-af16-00387d1791e9 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.446675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.447285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.447566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.447838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.448516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.450112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.450698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.451522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.451826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.452452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] PATH_INFO: `/v3/projects/ba66b134261b494b964e9f050449a0fe/users/1d22ff09fb6743c2880af0d7778fb984/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.459902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 798/1594] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.464046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.469841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba66b134261b494b964e9f050449a0fe, user_id=1d22ff09fb6743c2880af0d7778fb984, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.472576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.473337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a3823-49e2-47cb-8f14-380072edda76 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.478549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.479143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.479420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.479691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.480363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.481919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.487049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 797/1595] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:30 2022] PUT /identity/v3/projects/ba66b134261b494b964e9f050449a0fe/users/1d22ff09fb6743c2880af0d7778fb984/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:30.488227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.492990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 799/1596] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.494020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.500249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:30.504230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f882e5e-df6c-42ca-94f1-828f2f6b59ef admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.555600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:30.559911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c03e1-fe41-4ecf-b064-23c68908e3af admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.565479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4ad35-76f9-41f2-8874-c77e23e24749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.566251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4ad35-76f9-41f2-8874-c77e23e24749 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.566464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4ad35-76f9-41f2-8874-c77e23e24749 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.566753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4ad35-76f9-41f2-8874-c77e23e24749 admin admin] PATH_INFO: `/v3/projects/990ac78d52b04378986c13ab1992fef6/users/7bfab75cc95346f79015594fea0ab93e/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.577355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.577956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.578233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.578495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.579129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.580738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.581685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4ad35-76f9-41f2-8874-c77e23e24749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990ac78d52b04378986c13ab1992fef6, user_id=7bfab75cc95346f79015594fea0ab93e, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.584422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4ad35-76f9-41f2-8874-c77e23e24749 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.590835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 800/1597] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.591725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.593629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:30.604638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 798/1598] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:30 2022] PUT /identity/v3/projects/990ac78d52b04378986c13ab1992fef6/users/7bfab75cc95346f79015594fea0ab93e/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:30.606104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f137c65-548a-4c54-8501-a77bb60dee48 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.606417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f137c65-548a-4c54-8501-a77bb60dee48 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.606597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f137c65-548a-4c54-8501-a77bb60dee48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.636455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4f137c65-548a-4c54-8501-a77bb60dee48 None None] MFA Rules not processed for user `a2b91a1c4acc4cc88cd98a3215ebcc98`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:30.637741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4f137c65-548a-4c54-8501-a77bb60dee48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:30.658417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c5bf9-4df7-46fb-98a5-101bc8e3e7dc admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.707397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 799/1599] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:30.707397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f137c65-548a-4c54-8501-a77bb60dee48 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.714312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.715215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.715588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.715858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.772056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f137c65-548a-4c54-8501-a77bb60dee48 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.777079 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048a3bd-7002-4082-b637-8f36f45361ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.777747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048a3bd-7002-4082-b637-8f36f45361ac admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.778027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048a3bd-7002-4082-b637-8f36f45361ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.778346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048a3bd-7002-4082-b637-8f36f45361ac admin admin] PATH_INFO: `/v3/projects/6d0f2821338a4628882790d108bfe02c/users/b21bed13f7594ecbaf2194172abcfe8c/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.778443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.779785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.789127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 801/1600] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:30.789969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.793079 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048a3bd-7002-4082-b637-8f36f45361ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0f2821338a4628882790d108bfe02c, user_id=b21bed13f7594ecbaf2194172abcfe8c, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.795768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048a3bd-7002-4082-b637-8f36f45361ac admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.799466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815f37-8b72-4bbf-af9f-1806b1391da9 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.804543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.805219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.805595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.805884 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.806579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.808183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.813701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 800/1601] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:30 2022] PUT /identity/v3/projects/6d0f2821338a4628882790d108bfe02c/users/b21bed13f7594ecbaf2194172abcfe8c/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:30.815993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290305-a5e6-4f64-9b1a-ff71940cbf78 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.816446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290305-a5e6-4f64-9b1a-ff71940cbf78 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.816875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290305-a5e6-4f64-9b1a-ff71940cbf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.822970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 802/1602] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:30.822970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.832251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b047ce-eea6-4018-ab32-22496172f99d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.836446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:30.843588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e0290305-a5e6-4f64-9b1a-ff71940cbf78 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:30.845482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e0290305-a5e6-4f64-9b1a-ff71940cbf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:30.867514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:30.901656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a9d65-d160-4025-93bb-cb1f19a5c26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:30.902421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a9d65-d160-4025-93bb-cb1f19a5c26e admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.902758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a9d65-d160-4025-93bb-cb1f19a5c26e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.903050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a9d65-d160-4025-93bb-cb1f19a5c26e admin admin] PATH_INFO: `/v3/projects/d255b03b31004c3b8e446f318cd72265/users/68288f4e2f57454fbb06eff0dd72cca3/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.914620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 801/1603] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:30.916101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290305-a5e6-4f64-9b1a-ff71940cbf78 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:30.918899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a9d65-d160-4025-93bb-cb1f19a5c26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d255b03b31004c3b8e446f318cd72265, user_id=68288f4e2f57454fbb06eff0dd72cca3, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:30.921630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a9d65-d160-4025-93bb-cb1f19a5c26e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:30.931703 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290305-a5e6-4f64-9b1a-ff71940cbf78 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:30.937335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 803/1604] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:30 2022] PUT /identity/v3/projects/d255b03b31004c3b8e446f318cd72265/users/68288f4e2f57454fbb06eff0dd72cca3/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:30.938593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61583-f460-4581-8a6d-a244330531e3 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:30.938871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61583-f460-4581-8a6d-a244330531e3 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:30.939144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61583-f460-4581-8a6d-a244330531e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:30.978912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-15b61583-f460-4581-8a6d-a244330531e3 None None] MFA Rules not processed for user `7bfab75cc95346f79015594fea0ab93e`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:30.980586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-15b61583-f460-4581-8a6d-a244330531e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:31.000709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b50de6-7078-4891-a735-ab7738f35f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.001368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50de6-7078-4891-a735-ab7738f35f10 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.001651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50de6-7078-4891-a735-ab7738f35f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.001926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50de6-7078-4891-a735-ab7738f35f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.002588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b50de6-7078-4891-a735-ab7738f35f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.004466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b50de6-7078-4891-a735-ab7738f35f10 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.016736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 802/1605] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:31.018108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3b39-65e3-44fd-abd9-9c1076c5f7be None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.018396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3b39-65e3-44fd-abd9-9c1076c5f7be None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.018670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3b39-65e3-44fd-abd9-9c1076c5f7be None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.020298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 803/1606] 10.222.0.44 () {54 vars in 961 bytes} [Tue Oct 4 08:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:51:31.021033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3b39-65e3-44fd-abd9-9c1076c5f7be None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.035989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3b39-65e3-44fd-abd9-9c1076c5f7be None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.041537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.042184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.042440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.042712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] PATH_INFO: `/v3/projects/91d94cf00a1f41c7881a9e222a821cc8/users/9e28fc8794894af2a7d4459318f61abf/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.059380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d94cf00a1f41c7881a9e222a821cc8, user_id=9e28fc8794894af2a7d4459318f61abf, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.062101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.062262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 804/1607] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:31.063600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4da5-9e7f-4c48-9278-6a2abd0cc87c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.063993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4da5-9e7f-4c48-9278-6a2abd0cc87c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.064613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4da5-9e7f-4c48-9278-6a2abd0cc87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.081742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 804/1608] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:31 2022] PUT /identity/v3/projects/91d94cf00a1f41c7881a9e222a821cc8/users/9e28fc8794894af2a7d4459318f61abf/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:31.082559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.104977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3cfd4da5-9e7f-4c48-9278-6a2abd0cc87c None None] MFA Rules not processed for user `b21bed13f7594ecbaf2194172abcfe8c`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:31.106094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3cfd4da5-9e7f-4c48-9278-6a2abd0cc87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:31.151449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91ac28-2012-43fa-a60f-bb272f49d59c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.156465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.157568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.158366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.158926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.181052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 805/1609] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:31.182559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bbc4-b68e-4539-ba42-133dfd9bb57f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.182851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bbc4-b68e-4539-ba42-133dfd9bb57f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.183129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bbc4-b68e-4539-ba42-133dfd9bb57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.201934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:31.209118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-05e6bbc4-b68e-4539-ba42-133dfd9bb57f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:31.210557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-05e6bbc4-b68e-4539-ba42-133dfd9bb57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:31.251344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:31.258608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.260739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.280114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 805/1610] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:31.281623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.294806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd95346-6cad-4c91-ab4d-972a746534b6 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.297292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 806/1611] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:31.298682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e685c0f-f7f1-42be-b5e5-51e3803872c8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.298972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e685c0f-f7f1-42be-b5e5-51e3803872c8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.299242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e685c0f-f7f1-42be-b5e5-51e3803872c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.303579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.304422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.304721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.305067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] PATH_INFO: `/v3/projects/66a8ff125c714281a280f07c87209062/users/1be3a88197184a2ea2cc7a362c44935c/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.324736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a8ff125c714281a280f07c87209062, user_id=1be3a88197184a2ea2cc7a362c44935c, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.327696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.346060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 806/1612] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:31 2022] PUT /identity/v3/projects/66a8ff125c714281a280f07c87209062/users/1be3a88197184a2ea2cc7a362c44935c/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:31.347035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.357147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1e685c0f-f7f1-42be-b5e5-51e3803872c8 None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:31.358518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1e685c0f-f7f1-42be-b5e5-51e3803872c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:31.437664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a84fe-cf72-4e70-abe0-b8edf162d0fa admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.445411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcab7b9-95f2-437f-953e-9cf59d637963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.445848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcab7b9-95f2-437f-953e-9cf59d637963 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.446286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcab7b9-95f2-437f-953e-9cf59d637963 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.446609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcab7b9-95f2-437f-953e-9cf59d637963 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.447319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcab7b9-95f2-437f-953e-9cf59d637963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.453083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 807/1613] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:31.453762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcab7b9-95f2-437f-953e-9cf59d637963 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.455315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b785bbb-7943-410c-a00b-66474121e17d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.455772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b785bbb-7943-410c-a00b-66474121e17d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.456237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b785bbb-7943-410c-a00b-66474121e17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.491466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-8b785bbb-7943-410c-a00b-66474121e17d None None] MFA Rules not processed for user `9e28fc8794894af2a7d4459318f61abf`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:31.493264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8b785bbb-7943-410c-a00b-66474121e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:31.495392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 807/1614] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:31.497036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252184b9-1db7-4783-99e7-e7459f864e06 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.497342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252184b9-1db7-4783-99e7-e7459f864e06 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.497618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252184b9-1db7-4783-99e7-e7459f864e06 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.499246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 808/1615] 10.222.0.44 () {54 vars in 961 bytes} [Tue Oct 4 08:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:51:31.500032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252184b9-1db7-4783-99e7-e7459f864e06 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.575515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252184b9-1db7-4783-99e7-e7459f864e06 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.578748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 808/1616] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:31.581879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4eefa-d028-462a-9cd3-ec6eb8f23b99 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.582170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4eefa-d028-462a-9cd3-ec6eb8f23b99 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.582441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4eefa-d028-462a-9cd3-ec6eb8f23b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.584841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.585704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.586124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.586545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.616826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-83d4eefa-d028-462a-9cd3-ec6eb8f23b99 None None] MFA Rules not processed for user `1be3a88197184a2ea2cc7a362c44935c`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:31.619109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-83d4eefa-d028-462a-9cd3-ec6eb8f23b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:31.651648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:31.657837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:31.679062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.680623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.691613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 809/1617] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:31.702204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.708225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 809/1618] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:31.709321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4eefa-d028-462a-9cd3-ec6eb8f23b99 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.724916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4eefa-d028-462a-9cd3-ec6eb8f23b99 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.736279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad44a3-433b-4a79-afa7-6446471a0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.737115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad44a3-433b-4a79-afa7-6446471a0824 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.737964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad44a3-433b-4a79-afa7-6446471a0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.738936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad44a3-433b-4a79-afa7-6446471a0824 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.739752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad44a3-433b-4a79-afa7-6446471a0824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.742699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad44a3-433b-4a79-afa7-6446471a0824 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.779390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 810/1619] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/users => generated 289 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:31.780700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc31b9f-b5b9-4080-8681-11fe9a7da5b8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.780992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc31b9f-b5b9-4080-8681-11fe9a7da5b8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.781269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc31b9f-b5b9-4080-8681-11fe9a7da5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.812553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-dbc31b9f-b5b9-4080-8681-11fe9a7da5b8 None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:31.814276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-dbc31b9f-b5b9-4080-8681-11fe9a7da5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:31.829582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 811/1620] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:31.830000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc31b9f-b5b9-4080-8681-11fe9a7da5b8 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.836990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23815d16-457a-4a73-9b9c-c7688b481382 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.844575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc31b9f-b5b9-4080-8681-11fe9a7da5b8 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:31.851296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.852067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.852475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.852791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.853588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:31.855172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:31.855866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:31.856183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:31.856464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:31.856812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:31.910211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 812/1621] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:31 2022] POST /identity/v3/projects => generated 393 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:31.913289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:31.996275 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.004094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.026403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 810/1622] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:32.028473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.045388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f92046-a11a-4273-90e6-e23351b658b7 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.051675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.052388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.053059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.053190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.053952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.054915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffaf0d-abf6-40c2-8092-78c1e834b5ac service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.055684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.071282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 813/1623] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:32.072430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.084424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f31c-15ab-4eac-823a-65b27f468770 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.090421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.091064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.091385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.092158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.092638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.095038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.127409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 814/1624] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:32 2022] POST /identity/v3/users => generated 303 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:32.127409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.138650 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c286f-ee41-4c58-9119-2cb4c65289dd admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.143514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.144324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.144593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.144864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] PATH_INFO: `/v3/projects/74ca80207eda4415a3362e9caaed5317/users/3c7566c43931428db886b9485b7225af/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.162855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=74ca80207eda4415a3362e9caaed5317, user_id=3c7566c43931428db886b9485b7225af, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.166844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.187705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 815/1625] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:32 2022] PUT /identity/v3/projects/74ca80207eda4415a3362e9caaed5317/users/3c7566c43931428db886b9485b7225af/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:32.188854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.189542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.189857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.190084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.192382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.219931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:32.300948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:32.332393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.338486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.346474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28e78c-6ce2-46d4-bfdc-e29014b06f5d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.350046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 811/1626] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:32.350949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.364123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ef392-da9b-4bb1-931d-c46d6a4f3a26 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.366384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d41cb-ca57-47e3-a926-4d988b50b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.367278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d41cb-ca57-47e3-a926-4d988b50b13a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.368538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d41cb-ca57-47e3-a926-4d988b50b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.369944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d41cb-ca57-47e3-a926-4d988b50b13a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.370675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb9cfc-5e77-405e-896c-0deda8475d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.371369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d41cb-ca57-47e3-a926-4d988b50b13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.371687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb9cfc-5e77-405e-896c-0deda8475d43 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.372153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb9cfc-5e77-405e-896c-0deda8475d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.372589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb9cfc-5e77-405e-896c-0deda8475d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.373403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb9cfc-5e77-405e-896c-0deda8475d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.379998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d41cb-ca57-47e3-a926-4d988b50b13a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.384896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb9cfc-5e77-405e-896c-0deda8475d43 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.402410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 812/1627] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:32.408103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265c0c0-dc48-4b99-80f1-e2fddbb85721 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.408605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265c0c0-dc48-4b99-80f1-e2fddbb85721 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.409072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265c0c0-dc48-4b99-80f1-e2fddbb85721 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.411077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 813/1628] 10.222.0.44 () {54 vars in 912 bytes} [Tue Oct 4 08:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:51:32.412289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265c0c0-dc48-4b99-80f1-e2fddbb85721 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.423366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 816/1629] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:32.427039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653230e4-197e-42c5-b54d-37476942d1d6 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.427337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653230e4-197e-42c5-b54d-37476942d1d6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.429166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265c0c0-dc48-4b99-80f1-e2fddbb85721 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.432069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653230e4-197e-42c5-b54d-37476942d1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.434991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.435985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.436428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.436860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] PATH_INFO: `/v3/projects/74ca80207eda4415a3362e9caaed5317/users/3c7566c43931428db886b9485b7225af/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.440186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74ca80207eda4415a3362e9caaed5317, user_id=3c7566c43931428db886b9485b7225af, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.443340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.468346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 814/1630] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:32 2022] PUT /identity/v3/projects/74ca80207eda4415a3362e9caaed5317/users/3c7566c43931428db886b9485b7225af/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:32.470030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.502491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-653230e4-197e-42c5-b54d-37476942d1d6 None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:32.505045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-653230e4-197e-42c5-b54d-37476942d1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:32.577524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00206cbb-7dbd-48b7-85fe-78aed79729d7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.593591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.594925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.595938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.597109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] PATH_INFO: `/v3/projects/482abe2cc32c4191bd7d5012c3c5f9ac/users/3569e72d202842a586ff8503446f1d83/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.644341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=482abe2cc32c4191bd7d5012c3c5f9ac, user_id=3569e72d202842a586ff8503446f1d83, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.649981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.652058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 817/1631] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:51:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:32.653707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0481bf7-48c1-4ef4-a2a9-5caa40c6600f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.654125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0481bf7-48c1-4ef4-a2a9-5caa40c6600f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.654429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0481bf7-48c1-4ef4-a2a9-5caa40c6600f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.681147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 815/1632] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:32 2022] PUT /identity/v3/projects/482abe2cc32c4191bd7d5012c3c5f9ac/users/3569e72d202842a586ff8503446f1d83/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:32.682359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.724230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b0481bf7-48c1-4ef4-a2a9-5caa40c6600f None None] MFA Rules not processed for user `3c7566c43931428db886b9485b7225af`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:32.725562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b0481bf7-48c1-4ef4-a2a9-5caa40c6600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:32.769181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82873c-2a14-4d98-9c87-c0a08ee78fc2 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.773770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.774692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.775305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.775936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:32.810122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:32.869465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 818/1633] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:32.875523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0481bf7-48c1-4ef4-a2a9-5caa40c6600f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.912675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:32.914148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:32.950754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 816/1634] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:32.951710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:32.964599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed023e2-6782-4f70-bbb5-7d7c7b4e36d2 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:32.971985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:32.975716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e3989-6d53-444c-8b58-b84aab0f4945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:32.976706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3989-6d53-444c-8b58-b84aab0f4945 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:32.977062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3989-6d53-444c-8b58-b84aab0f4945 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:32.977950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3989-6d53-444c-8b58-b84aab0f4945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.003484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e3989-6d53-444c-8b58-b84aab0f4945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:33.004834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0481bf7-48c1-4ef4-a2a9-5caa40c6600f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:33.006647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e3989-6d53-444c-8b58-b84aab0f4945 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:33.018342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:33.019224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:33.019632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:33.020073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.020907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:33.022599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:33.023850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 817/1635] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:33.026710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf068-155e-4c65-b6e2-73161be5fe66 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:33.027191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf068-155e-4c65-b6e2-73161be5fe66 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:33.027615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf068-155e-4c65-b6e2-73161be5fe66 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.030250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 818/1636] 10.222.0.44 () {54 vars in 912 bytes} [Tue Oct 4 08:51:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:51:33.048505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 819/1637] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:33.048656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9268c-e54c-421b-974b-2c22d5ec7bef None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:33.052396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:33.052500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9268c-e54c-421b-974b-2c22d5ec7bef None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:33.052500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9268c-e54c-421b-974b-2c22d5ec7bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.065622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e8dea-667e-497d-a7bd-9f5d4ddfc8ec admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:33.071758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b1cf4-1b80-4c08-92d4-34a01216e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:33.072722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b1cf4-1b80-4c08-92d4-34a01216e3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:33.073126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b1cf4-1b80-4c08-92d4-34a01216e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:33.073534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b1cf4-1b80-4c08-92d4-34a01216e3d7 admin admin] PATH_INFO: `/v3/projects/482abe2cc32c4191bd7d5012c3c5f9ac/users/3569e72d202842a586ff8503446f1d83/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.075914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b1cf4-1b80-4c08-92d4-34a01216e3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=482abe2cc32c4191bd7d5012c3c5f9ac, user_id=3569e72d202842a586ff8503446f1d83, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:33.079259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b1cf4-1b80-4c08-92d4-34a01216e3d7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:33.107424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 820/1638] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:33 2022] PUT /identity/v3/projects/482abe2cc32c4191bd7d5012c3c5f9ac/users/3569e72d202842a586ff8503446f1d83/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:33.109125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316e6c4-03bc-4bd1-82f1-dc002386fbff None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:33.109492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316e6c4-03bc-4bd1-82f1-dc002386fbff None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:33.109777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316e6c4-03bc-4bd1-82f1-dc002386fbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.110936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-86e9268c-e54c-421b-974b-2c22d5ec7bef None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:33.113052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-86e9268c-e54c-421b-974b-2c22d5ec7bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:33.155334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-6316e6c4-03bc-4bd1-82f1-dc002386fbff None None] MFA Rules not processed for user `3569e72d202842a586ff8503446f1d83`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:33.156777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6316e6c4-03bc-4bd1-82f1-dc002386fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:33.210014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 819/1639] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:51:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:33.253769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9268c-e54c-421b-974b-2c22d5ec7bef None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:33.326599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 821/1640] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:33.356765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9268c-e54c-421b-974b-2c22d5ec7bef None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:33.360555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:33.361291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:33.361566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:33.361834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.375529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:33.377593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:33.403490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 820/1641] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:33.815074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316e6c4-03bc-4bd1-82f1-dc002386fbff None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:33.828164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316e6c4-03bc-4bd1-82f1-dc002386fbff None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:33.833328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:33.833952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:33.834238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:33.834506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:33.948270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:33.950157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:33.964884 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 822/1642] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:34.336210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:34.347448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32c39b-3515-4655-9b17-3b80264c7c7e service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:34.358566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:34.359350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:34.359719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:34.360120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:34.371429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:34.373515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:34.389747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 821/1643] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:34.437365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:34.452070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2d00d-85df-43bd-bcfd-4a41add62388 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:34.458595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:34.459220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:34.459503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:34.459815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:34.550598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:34.564842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d14382-da03-4ddd-b9c4-0c3c98dc4e95 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:34.577025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3925d-7947-467c-8bac-f0a957c1d03b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:34.577975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3925d-7947-467c-8bac-f0a957c1d03b service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:34.578456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3925d-7947-467c-8bac-f0a957c1d03b service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:34.578936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3925d-7947-467c-8bac-f0a957c1d03b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:34.636764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:34.644990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:34.646951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:34.652319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:34.663814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 823/1644] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:34.763828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3925d-7947-467c-8bac-f0a957c1d03b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:34.764613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3925d-7947-467c-8bac-f0a957c1d03b service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:34.802839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 822/1645] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:51:35.099095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:35.221029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8e1c4-db4e-4014-93f1-2d63783d377c service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:35.245407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:35.246512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:35.246993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:35.247462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] PATH_INFO: `/v3/users/9e28fc8794894af2a7d4459318f61abf` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:35.249872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e28fc8794894af2a7d4459318f61abf)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:35.251383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: WARNING py.warnings [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:51:35.251383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: warnings.warn(msg) Oct 04 08:51:35.251383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: Oct 04 08:51:35.253090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:35.464108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e28fc8794894af2a7d4459318f61abf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:35.469486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e28fc8794894af2a7d4459318f61abf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:35.470049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e28fc8794894af2a7d4459318f61abf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:35.500517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e28fc8794894af2a7d4459318f61abf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:35.556763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e28fc8794894af2a7d4459318f61abf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:35.592163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 824/1646] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:35 2022] DELETE /identity/v3/users/9e28fc8794894af2a7d4459318f61abf => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:35.596857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:35.597928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:35.598417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:35.660939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:35.662346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:35.896483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 823/1647] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:35.937536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:35.941352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:36.072243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4095ab1-01e5-44e3-9a18-8c26c7f28e1a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:36.078823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:36.079499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:36.079770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:36.080136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:36.101787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:36.103527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:36.130735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 825/1648] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:37.296646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:37.299087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:37.322612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc1793-f455-4f48-a29c-a38e9a6d1b46 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:37.338146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:37.339136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:37.339565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:37.340143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] PATH_INFO: `/v3/projects/91d94cf00a1f41c7881a9e222a821cc8` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:37.341688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d94cf00a1f41c7881a9e222a821cc8)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:37.342839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: WARNING py.warnings [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:51:37.342839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: warnings.warn(msg) Oct 04 08:51:37.342839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: Oct 04 08:51:37.344474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:37.482559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d94cf00a1f41c7881a9e222a821cc8', 'request_id': 'req-16248464-b8e9-4c2a-9e81-04ee35d76805'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:37.530785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d94cf00a1f41c7881a9e222a821cc8', 'request_id': 'req-16248464-b8e9-4c2a-9e81-04ee35d76805'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:37.564612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d94cf00a1f41c7881a9e222a821cc8', 'request_id': 'req-16248464-b8e9-4c2a-9e81-04ee35d76805'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:37.581904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:37.588641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:37.605615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] The token cache is being invalidate because project 91d94cf00a1f41c7881a9e222a821cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:37.606316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:37.611267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-16248464-b8e9-4c2a-9e81-04ee35d76805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:37.620090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 824/1649] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:37 2022] DELETE /identity/v3/projects/91d94cf00a1f41c7881a9e222a821cc8 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:37.623769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d3ead-ac83-488f-bcf3-beace0b1902a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:37.625164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d3ead-ac83-488f-bcf3-beace0b1902a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:37.626550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d3ead-ac83-488f-bcf3-beace0b1902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:37.627314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357194-e262-4f2b-b47d-52c8f77234d9 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:37.688740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:37.698263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-f70d3ead-ac83-488f-bcf3-beace0b1902a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:37.706240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f70d3ead-ac83-488f-bcf3-beace0b1902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:37.794817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:37.795602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:37.796050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:37.796477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] PATH_INFO: `/v3/users/a2b91a1c4acc4cc88cd98a3215ebcc98` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:37.798881 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b91a1c4acc4cc88cd98a3215ebcc98)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:37.809106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:37.879242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:37.978293 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 825/1650] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:37.979937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728b713-7d81-4036-9f88-c8d32a2d9ccd None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:37.980629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728b713-7d81-4036-9f88-c8d32a2d9ccd None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:37.980629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728b713-7d81-4036-9f88-c8d32a2d9ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:37.988474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b91a1c4acc4cc88cd98a3215ebcc98'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:37.989689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b91a1c4acc4cc88cd98a3215ebcc98'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:37.990755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b91a1c4acc4cc88cd98a3215ebcc98'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:38.026292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b91a1c4acc4cc88cd98a3215ebcc98'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:38.035517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4728b713-7d81-4036-9f88-c8d32a2d9ccd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:38.039169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4728b713-7d81-4036-9f88-c8d32a2d9ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:38.058019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c83232b4-a16f-492b-baf4-5e1fb72c5e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b91a1c4acc4cc88cd98a3215ebcc98'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:38.075689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 826/1651] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:51:38.081303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728b713-7d81-4036-9f88-c8d32a2d9ccd None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:38.099878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 826/1652] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:37 2022] DELETE /identity/v3/users/a2b91a1c4acc4cc88cd98a3215ebcc98 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:51:38.104379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:38.104674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:38.104945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:38.119577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728b713-7d81-4036-9f88-c8d32a2d9ccd None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:38.155485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:38.156944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:38.309029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 827/1653] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:38.347869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:38.365405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:38.374580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:38.375228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:38.375501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:38.376062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:38.376887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:38.379419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:38.386599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:38.427769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 827/1654] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:38.428859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:38.465503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515cf5d-bf08-48d9-a13b-b13651aeeb9c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:38.481465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:38.486454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:38.487099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:38.492001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:38.492307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:38.494305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:38.499138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:38.517941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afe438-22d7-4484-8cf5-345a1435bf4f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:38.529592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:38.530254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:38.530537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:38.530814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:38.554823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:38.556386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:38.589574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 828/1655] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:38.600117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 828/1656] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:38 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:38.606712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:38.624362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20748a-af81-4e21-ac05-86a55be1d9f3 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:38.780733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:38.781639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:38.781989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:38.782321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:38.783097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:38.785274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:38.856128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 829/1657] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:38 2022] POST /identity/v3/users => generated 287 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:38.860994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:38.886384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aade0d9-9e3c-47fd-a463-94a3b74f8516 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:38.904199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:38.904842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:38.905125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:38.905400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:38.918982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:38.924321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:38.976143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 829/1658] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:38.980543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:38.999624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbaf62-3e4f-4247-8cbe-2262a25e857c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:39.014330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180d1df-7956-4a31-85e3-4602cfa978f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:39.016258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180d1df-7956-4a31-85e3-4602cfa978f5 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:39.017874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180d1df-7956-4a31-85e3-4602cfa978f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:39.018281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180d1df-7956-4a31-85e3-4602cfa978f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:39.020012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180d1df-7956-4a31-85e3-4602cfa978f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:39.025804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180d1df-7956-4a31-85e3-4602cfa978f5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:39.060009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 830/1659] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:39.061461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:39.076301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27de22-b7fb-474c-a907-f5afb0cca315 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:39.091195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:39.091955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:39.092237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:39.092506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] PATH_INFO: `/v3/projects/f0bbff82c56e4c84874a4d8fc964af00/users/b0e38b63fb94447d86997507f9f2e4e3/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:39.132362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0bbff82c56e4c84874a4d8fc964af00, user_id=b0e38b63fb94447d86997507f9f2e4e3, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:39.139698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:39.166847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 830/1660] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:39 2022] PUT /identity/v3/projects/f0bbff82c56e4c84874a4d8fc964af00/users/b0e38b63fb94447d86997507f9f2e4e3/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:39.180677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aa52c-b254-4061-b424-2be842d5dc0c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:39.181002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aa52c-b254-4061-b424-2be842d5dc0c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:39.181279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aa52c-b254-4061-b424-2be842d5dc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:39.230829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-022aa52c-b254-4061-b424-2be842d5dc0c None None] MFA Rules not processed for user `b0e38b63fb94447d86997507f9f2e4e3`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:39.233900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-022aa52c-b254-4061-b424-2be842d5dc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:39.392104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 831/1661] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:39.395010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:39.552999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9da3c-d7b3-4a4c-94a2-eab465a756bf admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:39.582172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:39.583134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:39.583542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:39.583938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:39.584858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:39.587006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:39.663968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:39.685745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 831/1662] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:39 2022] POST /identity/v3/projects => generated 371 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:39.691347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022aa52c-b254-4061-b424-2be842d5dc0c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:39.714091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022aa52c-b254-4061-b424-2be842d5dc0c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:39.799522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:39.817637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:39.818561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:39.819144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:39.819551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:39.820500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:39.822646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:39.884104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 832/1663] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:39 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:39.885524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:39.903991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dd119-3257-4a25-9527-bff629f0c812 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:39.920305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:39.920948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:39.921225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:39.921494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:39.922152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:39.923766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:39.950672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 832/1664] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:39.952080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:39.966493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c218f-6e8a-4529-b712-37b1bd5b956c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:39.984987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:39.986809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:39.987093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:39.987701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] PATH_INFO: `/v3/projects/4a9a16c93b6f4662bf2303884bd0fb71/users/d8f73b49a8e8482c9dbec8fcec09b3b0/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.063563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9a16c93b6f4662bf2303884bd0fb71, user_id=d8f73b49a8e8482c9dbec8fcec09b3b0, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.082012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:40.112839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:40.114771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:40.122946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 833/1665] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:39 2022] PUT /identity/v3/projects/4a9a16c93b6f4662bf2303884bd0fb71/users/d8f73b49a8e8482c9dbec8fcec09b3b0/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:40.128873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:40.147965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e124b2-acc4-4541-96be-847eb28376a3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:40.326909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7698d8-4016-422c-a322-bcd78caeb870 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:40.339662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:40.340482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.340827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.341186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] PATH_INFO: `/v3/projects/269445aa5f0745a4ac2eff02524fbcc0` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.341859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:40.342713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=269445aa5f0745a4ac2eff02524fbcc0)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.342789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.343058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.343419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.344266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.345966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:40.348841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:40.392266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 834/1666] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:40.394640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:40.416028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cfa79-1e96-4d20-92ce-9a62190bf621 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:40.433292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb125f-2d42-4107-8c19-6fc3a06657bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:40.434403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb125f-2d42-4107-8c19-6fc3a06657bf admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.435113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb125f-2d42-4107-8c19-6fc3a06657bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.435397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb125f-2d42-4107-8c19-6fc3a06657bf admin admin] PATH_INFO: `/v3/projects/4a9a16c93b6f4662bf2303884bd0fb71/users/d8f73b49a8e8482c9dbec8fcec09b3b0/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.444050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb125f-2d42-4107-8c19-6fc3a06657bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9a16c93b6f4662bf2303884bd0fb71, user_id=d8f73b49a8e8482c9dbec8fcec09b3b0, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.448429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb125f-2d42-4107-8c19-6fc3a06657bf admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:40.476780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 835/1667] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:40 2022] PUT /identity/v3/projects/4a9a16c93b6f4662bf2303884bd0fb71/users/d8f73b49a8e8482c9dbec8fcec09b3b0/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:51:40.487072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.487970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.488278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.510002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '269445aa5f0745a4ac2eff02524fbcc0', 'request_id': 'req-051dc60f-2204-4a2d-99de-b2fa151ceb21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:40.518794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '269445aa5f0745a4ac2eff02524fbcc0', 'request_id': 'req-051dc60f-2204-4a2d-99de-b2fa151ceb21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:40.530193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '269445aa5f0745a4ac2eff02524fbcc0', 'request_id': 'req-051dc60f-2204-4a2d-99de-b2fa151ceb21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:40.564402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] The token cache is being invalidate because project 269445aa5f0745a4ac2eff02524fbcc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:40.564779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:40.565541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-051dc60f-2204-4a2d-99de-b2fa151ceb21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:40.571737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 833/1668] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:40 2022] DELETE /identity/v3/projects/269445aa5f0745a4ac2eff02524fbcc0 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:40.573014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03dd4-fe37-4816-a6b8-94534a292e59 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.573367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03dd4-fe37-4816-a6b8-94534a292e59 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.573715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03dd4-fe37-4816-a6b8-94534a292e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.582791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] MFA Rules not processed for user `d8f73b49a8e8482c9dbec8fcec09b3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:40.584051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:40.605166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-1da03dd4-fe37-4816-a6b8-94534a292e59 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:40.606894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1da03dd4-fe37-4816-a6b8-94534a292e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:40.735484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 834/1669] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:40.746621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2824fc-a7df-4457-9f54-52cf962490d8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.746910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2824fc-a7df-4457-9f54-52cf962490d8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.747171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2824fc-a7df-4457-9f54-52cf962490d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.764966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 836/1670] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:51:40.773327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:40.775295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:40.791027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-dc2824fc-a7df-4457-9f54-52cf962490d8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:40.793509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-dc2824fc-a7df-4457-9f54-52cf962490d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:40.830473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 835/1671] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:40.831482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2824fc-a7df-4457-9f54-52cf962490d8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:40.847124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:40.856413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2824fc-a7df-4457-9f54-52cf962490d8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:40.868814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:40.872511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.872791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.873065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.873749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:40.874051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.879569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:40.903823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:40.917760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 836/1672] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:40.919423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:40.932716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c65c-9f20-4f7c-a391-59c7b0c3862b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:40.941889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2e8b8-22b1-4f95-bb77-49c3a5d2abfd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:40.943751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:40.944419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.944755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.945033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.952583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:40.953727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.954385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.954841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.955669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.957973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:40.970691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.974575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:41.002821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 837/1673] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:51:41.014412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 837/1674] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:40 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:41.014929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:41.036926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3d8ea-c336-41c6-8947-ecfdc97f4084 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:41.144467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:41.145974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.147135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.147733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.147856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:41.148860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:41.151041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:41.191154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 838/1675] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:41 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:51:41.192971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:41.216997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d0b2-a010-4087-9baf-4a57c4403a36 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:41.233675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:41.237791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.238077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.238358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.239063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:41.246073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:41.288235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 839/1676] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:41.289550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:41.306862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0258d-a857-4f25-836c-91d87d8ee93f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:41.313014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9122ff-7e52-4e37-8e9e-69bccbd84719 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:41.319371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:41.320141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.320572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.325045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.331468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa40e63-99aa-4f89-9a45-009d39a4b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:41.332168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa40e63-99aa-4f89-9a45-009d39a4b277 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.333993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:41.338783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa40e63-99aa-4f89-9a45-009d39a4b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.339138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa40e63-99aa-4f89-9a45-009d39a4b277 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.339847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa40e63-99aa-4f89-9a45-009d39a4b277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:41.341503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa40e63-99aa-4f89-9a45-009d39a4b277 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:41.378527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 840/1677] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:41.385017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9423cc6-9554-472d-9688-32088628640d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.385384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9423cc6-9554-472d-9688-32088628640d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.385661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9423cc6-9554-472d-9688-32088628640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.451631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c9423cc6-9554-472d-9688-32088628640d None None] MFA Rules not processed for user `91ccd41938954a36adb45c70e6ff41de`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:41.456956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:41.459486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:41.462035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c9423cc6-9554-472d-9688-32088628640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:41.486448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 838/1678] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:51:41.487983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:41.517182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90687df2-6314-4784-b0b5-546b5dd99977 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:41.533773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778800a0-9287-4753-862d-37172bcdd6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:41.534860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778800a0-9287-4753-862d-37172bcdd6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.536983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778800a0-9287-4753-862d-37172bcdd6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.537346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778800a0-9287-4753-862d-37172bcdd6ef admin admin] PATH_INFO: `/v3/projects/ac7c04a627f640e2ae09b70157e93639/users/9e9aed2496514805baa81c8d0a553184/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.582461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778800a0-9287-4753-862d-37172bcdd6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7c04a627f640e2ae09b70157e93639, user_id=9e9aed2496514805baa81c8d0a553184, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:41.590601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778800a0-9287-4753-862d-37172bcdd6ef admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:41.626883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 839/1679] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:41 2022] PUT /identity/v3/projects/ac7c04a627f640e2ae09b70157e93639/users/9e9aed2496514805baa81c8d0a553184/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:41.638428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe74769-1546-4848-8f21-fc7aaef53f0d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.638880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe74769-1546-4848-8f21-fc7aaef53f0d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.639161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe74769-1546-4848-8f21-fc7aaef53f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.667057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 841/1680] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:41.682017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:41.682633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:41.683452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:41.718605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-7fe74769-1546-4848-8f21-fc7aaef53f0d None None] MFA Rules not processed for user `9e9aed2496514805baa81c8d0a553184`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:41.719857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7fe74769-1546-4848-8f21-fc7aaef53f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:41.724804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] MFA Rules not processed for user `91ccd41938954a36adb45c70e6ff41de`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:41.726426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:41.800304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:41.900570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 842/1681] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:41.912698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:41.914193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:41.916740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:41.923789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 840/1682] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:41 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:41.925119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe74769-1546-4848-8f21-fc7aaef53f0d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:42.050646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53f94f-0d0b-4b54-b908-6b4247a352f0 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:42.053185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe74769-1546-4848-8f21-fc7aaef53f0d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:42.056759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92457f03-b445-409e-8831-731663f25b8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:42.057405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457f03-b445-409e-8831-731663f25b8f service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.057680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457f03-b445-409e-8831-731663f25b8f service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.058042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457f03-b445-409e-8831-731663f25b8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.065718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:42.067002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.067282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.067565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.068549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:42.068921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92457f03-b445-409e-8831-731663f25b8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:42.070506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:42.070609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92457f03-b445-409e-8831-731663f25b8f service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:42.094239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 843/1683] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:42.132743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 841/1684] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:41 2022] POST /identity/v3/projects => generated 405 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:42.135670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92457f03-b445-409e-8831-731663f25b8f service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:42.150768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92457f03-b445-409e-8831-731663f25b8f service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:42.267274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:42.267917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.268231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.268500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.269133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:42.271056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:42.310075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 844/1685] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:42 2022] POST /identity/v3/users => generated 309 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:42.310940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:42.325007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465183d-c6fa-43a8-b195-48b9f92484ee admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:42.333431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:42.334445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.334960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.335608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.336394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:42.338349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:42.355215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 842/1686] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:42.357054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:42.387619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19afec-3e23-4bb8-aa90-89ab1d4e5766 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:42.400799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:42.412372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.412669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.413504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] PATH_INFO: `/v3/projects/3359528402bc4f7cb406f608c9982436/users/428545e4e3394a80a3a476c987f797b5/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.433385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3359528402bc4f7cb406f608c9982436, user_id=428545e4e3394a80a3a476c987f797b5, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:42.436955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:42.456506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 845/1687] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:42 2022] PUT /identity/v3/projects/3359528402bc4f7cb406f608c9982436/users/428545e4e3394a80a3a476c987f797b5/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:42.460799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:42.473524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852895b-6251-460d-9220-9962a65134b1 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:42.639861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c9469-58b2-43b9-8df7-be1d4a284f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:42.640716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c9469-58b2-43b9-8df7-be1d4a284f1a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.641005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c9469-58b2-43b9-8df7-be1d4a284f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.641278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c9469-58b2-43b9-8df7-be1d4a284f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.641933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c9469-58b2-43b9-8df7-be1d4a284f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:42.643698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c9469-58b2-43b9-8df7-be1d4a284f1a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:42.670851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 843/1688] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:42.671070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:42.685415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddbfff-f6ed-4b1a-9626-46dfcc7dfc21 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:42.700428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1985a14-43b5-4a9c-b4cf-b0c180384b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:42.701319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1985a14-43b5-4a9c-b4cf-b0c180384b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.701595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1985a14-43b5-4a9c-b4cf-b0c180384b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.701899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1985a14-43b5-4a9c-b4cf-b0c180384b0d admin admin] PATH_INFO: `/v3/projects/3359528402bc4f7cb406f608c9982436/users/428545e4e3394a80a3a476c987f797b5/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.703908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1985a14-43b5-4a9c-b4cf-b0c180384b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3359528402bc4f7cb406f608c9982436, user_id=428545e4e3394a80a3a476c987f797b5, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:42.706534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1985a14-43b5-4a9c-b4cf-b0c180384b0d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:42.737078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 846/1689] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:42 2022] PUT /identity/v3/projects/3359528402bc4f7cb406f608c9982436/users/428545e4e3394a80a3a476c987f797b5/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:42.748335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.748611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.748889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.817590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] MFA Rules not processed for user `428545e4e3394a80a3a476c987f797b5`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:42.819024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:42.950348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbc77-f2b5-436c-b084-f5c30c296168 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.950892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbc77-f2b5-436c-b084-f5c30c296168 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.950892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbc77-f2b5-436c-b084-f5c30c296168 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.952891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 847/1690] 10.222.0.44 () {54 vars in 912 bytes} [Tue Oct 4 08:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:51:42.963131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:42.963311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:42.963709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:42.996681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:43.001827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:43.027985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 844/1691] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:42 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:43.040321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:43.042849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:43.059358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] MFA Rules not processed for user `349ee49c297749468e09c4aa3dc9c218`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:43.060451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:43.272223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccbe92-0b7b-49e1-b90f-771198d53d02 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:43.287655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0340ac-2afe-4478-89d3-758cb3c72f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:43.288326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0340ac-2afe-4478-89d3-758cb3c72f32 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:43.288860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0340ac-2afe-4478-89d3-758cb3c72f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:43.289132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0340ac-2afe-4478-89d3-758cb3c72f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:43.290425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 848/1692] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:51:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:43.307990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:43.312600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:43.322884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0340ac-2afe-4478-89d3-758cb3c72f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:43.324396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0340ac-2afe-4478-89d3-758cb3c72f32 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:43.344503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b84a-c887-4bd5-9064-4f10def3ffad None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:43.350091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:43.351225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:43.351695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:43.352687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:43.364593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 845/1693] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:43 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:51:43.415267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:43.419150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:43.471107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 849/1694] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:43.975671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87678-02b7-4fcb-8b75-d968d51c6e38 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:43.976520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87678-02b7-4fcb-8b75-d968d51c6e38 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:43.976930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87678-02b7-4fcb-8b75-d968d51c6e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:44.045367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4dd87678-02b7-4fcb-8b75-d968d51c6e38 None None] MFA Rules not processed for user `1d22ff09fb6743c2880af0d7778fb984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:44.046967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4dd87678-02b7-4fcb-8b75-d968d51c6e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:44.153050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:44.155206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:44.246479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 846/1695] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:44.247644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87678-02b7-4fcb-8b75-d968d51c6e38 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:44.382010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030ca7-8882-416d-8329-9c780ca4c17e service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:44.391402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:44.392051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:44.392368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:44.392945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] PATH_INFO: `/v3/users/1be3a88197184a2ea2cc7a362c44935c` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:44.401656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be3a88197184a2ea2cc7a362c44935c)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:44.406907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:44.481365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87678-02b7-4fcb-8b75-d968d51c6e38 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:44.497270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:44.497977 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:44.498277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:44.498561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] PATH_INFO: `/v3/users/9e9aed2496514805baa81c8d0a553184` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:44.505276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9aed2496514805baa81c8d0a553184)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:44.522996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:44.606948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:44.614966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be3a88197184a2ea2cc7a362c44935c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.616162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be3a88197184a2ea2cc7a362c44935c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.622697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be3a88197184a2ea2cc7a362c44935c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.646790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be3a88197184a2ea2cc7a362c44935c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.687610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9aed2496514805baa81c8d0a553184'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.689308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9aed2496514805baa81c8d0a553184'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.695093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3e53501c-1cab-4aef-86c4-8c330ee1592b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be3a88197184a2ea2cc7a362c44935c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.710505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9aed2496514805baa81c8d0a553184'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.712210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9aed2496514805baa81c8d0a553184'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.724934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 850/1696] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:44 2022] DELETE /identity/v3/users/1be3a88197184a2ea2cc7a362c44935c => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:44.727151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:44.728374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:44.728787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:44.760694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] MFA Rules not processed for user `1d22ff09fb6743c2880af0d7778fb984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:44.762626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:44.762734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9aed2496514805baa81c8d0a553184'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:44.768931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:44.808480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 847/1697] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:44 2022] DELETE /identity/v3/users/9e9aed2496514805baa81c8d0a553184 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:44.808920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:44.815916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:44.950694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868e355-28c8-4ecc-97e8-53fd8b9545de admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:44.957870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:44.958610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:44.958928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:44.959362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] PATH_INFO: `/v3/users/7bfab75cc95346f79015594fea0ab93e` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:44.960880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bfab75cc95346f79015594fea0ab93e)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:44.963087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:44.972557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 851/1698] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:44.973734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:44.975284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.052572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bfab75cc95346f79015594fea0ab93e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:45.053160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bfab75cc95346f79015594fea0ab93e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:45.062657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bfab75cc95346f79015594fea0ab93e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:45.063393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bfab75cc95346f79015594fea0ab93e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:45.096056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-20c71eb2-e822-4978-9a6f-9fe71a40f670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bfab75cc95346f79015594fea0ab93e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:45.123564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 848/1699] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:44 2022] DELETE /identity/v3/users/7bfab75cc95346f79015594fea0ab93e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:45.127157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.127449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.127724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.139556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac25f92-b70f-4893-9f91-12783b2c456f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:45.179545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:45.181172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.230359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:45.293798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e6b6e-c750-448a-a324-38534434e31a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:45.294545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e6b6e-c750-448a-a324-38534434e31a service placement] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.294752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e6b6e-c750-448a-a324-38534434e31a service placement] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.295070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e6b6e-c750-448a-a324-38534434e31a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.306048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-855e6b6e-c750-448a-a324-38534434e31a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.341724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e6b6e-c750-448a-a324-38534434e31a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:45.352961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e6b6e-c750-448a-a324-38534434e31a service placement] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:45.401094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 852/1700] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:45.406237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 849/1701] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:45.406386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f396a-1eb2-4961-9fb9-6fc8a52b38ed None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.406472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:45.406578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f396a-1eb2-4961-9fb9-6fc8a52b38ed None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.407762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.407969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f396a-1eb2-4961-9fb9-6fc8a52b38ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.468707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-264f396a-1eb2-4961-9fb9-6fc8a52b38ed None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:45.472039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-264f396a-1eb2-4961-9fb9-6fc8a52b38ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.500453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 853/1702] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:45.505981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.506295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.506575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.532639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:45.533856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.543814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 854/1703] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:45.544872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:45.546983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.625385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10cdd2-514d-4869-8bca-dee3337797fe None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:45.639595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:45.641423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.641423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.641643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.646971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:45.649009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:45.682395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde591f-c4e8-4c50-b898-f07d8b3b2b5f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:45.693252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:45.698011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.698429 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.698871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.711747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:45.713354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:45.741729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 855/1704] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:45.742919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:45.754411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:45.758593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362235-429b-4e79-96d0-15c64be1ff04 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:45.760563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:45.764037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4cf6d-cc1d-47f2-a217-9e62520f6b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:45.764679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4cf6d-cc1d-47f2-a217-9e62520f6b70 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.765544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4cf6d-cc1d-47f2-a217-9e62520f6b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.765833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4cf6d-cc1d-47f2-a217-9e62520f6b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.782373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4cf6d-cc1d-47f2-a217-9e62520f6b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:45.785100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4cf6d-cc1d-47f2-a217-9e62520f6b70 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:45.793853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 850/1705] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:45.799163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:45.810776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 856/1706] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:45.824363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec1b13-821e-42db-86e1-61f5f871f375 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:45.871040 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:45.871714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:45.872023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:45.872340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:45.915838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:45.918725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:45.955168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 851/1707] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:46.187982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:46.188643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:46.189286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:46.264254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] MFA Rules not processed for user `68288f4e2f57454fbb06eff0dd72cca3`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:46.268040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:46.365742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 857/1708] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:46.380872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:46.382313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:46.411740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66be51-95d1-4494-aa46-8baaf87573be service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:46.425981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:46.426958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:46.428157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:46.429606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:46.460630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:46.466904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:46.499549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 852/1709] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:47.322456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:47.324264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:47.351734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c599-c67f-492d-866b-da3369c9e114 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:47.366264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:47.368176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:47.369120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:47.370244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] PATH_INFO: `/v3/projects/ac7c04a627f640e2ae09b70157e93639` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:47.374268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7c04a627f640e2ae09b70157e93639)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:47.379236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:47.488564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7c04a627f640e2ae09b70157e93639', 'request_id': 'req-e4846dd5-e0c6-4676-a640-e5fd918ef325'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.502708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7c04a627f640e2ae09b70157e93639', 'request_id': 'req-e4846dd5-e0c6-4676-a640-e5fd918ef325'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.525405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7c04a627f640e2ae09b70157e93639', 'request_id': 'req-e4846dd5-e0c6-4676-a640-e5fd918ef325'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.553509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] The token cache is being invalidate because project ac7c04a627f640e2ae09b70157e93639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:47.553852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.555984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.557864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 858/1710] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:47 2022] DELETE /identity/v3/projects/ac7c04a627f640e2ae09b70157e93639 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:47.561743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:47.577373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b729713-1386-4864-b2c4-e8cb5b520ef8 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:47.658483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:47.719758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:47.720820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:47.721385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:47.721817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] PATH_INFO: `/v3/users/428545e4e3394a80a3a476c987f797b5` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:47.723953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=428545e4e3394a80a3a476c987f797b5)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:47.745582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:47.806166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:47.807641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:47.840411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4846dd5-e0c6-4676-a640-e5fd918ef325 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:47.858201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:47.861817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:47.862746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:47.863730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] PATH_INFO: `/v3/projects/990ac78d52b04378986c13ab1992fef6` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:47.875352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990ac78d52b04378986c13ab1992fef6)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:47.877397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:47.927089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '428545e4e3394a80a3a476c987f797b5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.931914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '428545e4e3394a80a3a476c987f797b5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.978606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '428545e4e3394a80a3a476c987f797b5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:47.983089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '428545e4e3394a80a3a476c987f797b5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.052291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '428545e4e3394a80a3a476c987f797b5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.078107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:48.103409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 853/1711] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:47 2022] DELETE /identity/v3/users/428545e4e3394a80a3a476c987f797b5 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:48.106781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:48.112051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:48.118420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990ac78d52b04378986c13ab1992fef6', 'request_id': 'req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.139975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990ac78d52b04378986c13ab1992fef6', 'request_id': 'req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.164961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990ac78d52b04378986c13ab1992fef6', 'request_id': 'req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.198415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] The token cache is being invalidate because project 990ac78d52b04378986c13ab1992fef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:48.199041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.199958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba06d7e7-b53e-4a95-9ddd-9584570e6f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.212751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 859/1712] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:47 2022] DELETE /identity/v3/projects/990ac78d52b04378986c13ab1992fef6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:51:48.214742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f66f2-d958-4720-afff-442e42092c57 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:48.215353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f66f2-d958-4720-afff-442e42092c57 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:48.216510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f66f2-d958-4720-afff-442e42092c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:48.282111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861eab7-1dce-4459-ad62-4525c637a91e admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:48.289528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-046f66f2-d958-4720-afff-442e42092c57 None None] MFA Rules not processed for user `1d22ff09fb6743c2880af0d7778fb984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:48.293731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:48.296697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:48.297101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:48.297314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-046f66f2-d958-4720-afff-442e42092c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:48.297429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] PATH_INFO: `/v3/projects/66a8ff125c714281a280f07c87209062` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:48.299553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a8ff125c714281a280f07c87209062)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:48.304072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:48.446243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 860/1713] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:48.447300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f66f2-d958-4720-afff-442e42092c57 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:48.448772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-046f66f2-d958-4720-afff-442e42092c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:48.461482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a8ff125c714281a280f07c87209062', 'request_id': 'req-ce4f1712-a6eb-4bce-8334-13a45118fe44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.482583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f66f2-d958-4720-afff-442e42092c57 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:48.489979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a8ff125c714281a280f07c87209062', 'request_id': 'req-ce4f1712-a6eb-4bce-8334-13a45118fe44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.496642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:48.507928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a8ff125c714281a280f07c87209062', 'request_id': 'req-ce4f1712-a6eb-4bce-8334-13a45118fe44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.538776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] The token cache is being invalidate because project 66a8ff125c714281a280f07c87209062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:48.540515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.543814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.548944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 854/1714] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:48 2022] DELETE /identity/v3/projects/66a8ff125c714281a280f07c87209062 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:48.553527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:48.559080 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:48.610547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:48.611585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:48.612501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:48.613005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] PATH_INFO: `/v3/users/3c7566c43931428db886b9485b7225af` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:48.614710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7566c43931428db886b9485b7225af)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:48.619857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:48.673022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f1712-a6eb-4bce-8334-13a45118fe44 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:48.679392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:48.680355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:48.680630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:48.680896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] PATH_INFO: `/v3/users/3569e72d202842a586ff8503446f1d83` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:48.682047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3569e72d202842a586ff8503446f1d83)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:48.684103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:48.759849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7566c43931428db886b9485b7225af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.760915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7566c43931428db886b9485b7225af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.761804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7566c43931428db886b9485b7225af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.778174 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3569e72d202842a586ff8503446f1d83'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.779137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3569e72d202842a586ff8503446f1d83'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.782087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7566c43931428db886b9485b7225af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.794684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:48.812825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f47eb22a-66c5-4ed5-b592-fed2a0186d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7566c43931428db886b9485b7225af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.815619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3569e72d202842a586ff8503446f1d83'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.818398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3569e72d202842a586ff8503446f1d83'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.849189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6e4ec426-efd4-42cb-bc18-2082c1e4afa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3569e72d202842a586ff8503446f1d83'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:48.852955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 861/1715] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:48 2022] DELETE /identity/v3/users/3c7566c43931428db886b9485b7225af => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:48.892096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 855/1716] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:48 2022] DELETE /identity/v3/users/3569e72d202842a586ff8503446f1d83 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:49.238357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d2789-0ddd-409d-89f1-9388c099e824 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:49.238944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d2789-0ddd-409d-89f1-9388c099e824 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:49.239352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d2789-0ddd-409d-89f1-9388c099e824 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:49.245942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 862/1717] 10.222.0.44 () {54 vars in 912 bytes} [Tue Oct 4 08:51:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:51:49.251909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:49.252306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:49.252683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:49.311026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] MFA Rules not processed for user `349ee49c297749468e09c4aa3dc9c218`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:49.314473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:49.446895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 856/1718] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:51:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:51:49.457266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d2789-0ddd-409d-89f1-9388c099e824 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:49.460505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-cd3d2789-0ddd-409d-89f1-9388c099e824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:49.468428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:49.471916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:49.507509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d2789-0ddd-409d-89f1-9388c099e824 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:49.515850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:49.516519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:49.518567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:49.520254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:49.520940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:49.562519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:49.567156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:49.599809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 863/1719] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:51:49.698688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160c362-6b0f-4ea2-92e0-d9af29df7633 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:49.748515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:49.751765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:49.753352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:49.753846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] PATH_INFO: `/v3/projects/3359528402bc4f7cb406f608c9982436` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:49.760501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3359528402bc4f7cb406f608c9982436)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:49.770082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:49.879582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:49.881421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:49.907839 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b793294-ac92-40b0-8962-c216e7ecb771 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:49.909129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3359528402bc4f7cb406f608c9982436', 'request_id': 'req-9a3313af-d69b-4272-a458-886ba03a4240'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:49.925401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3359528402bc4f7cb406f608c9982436', 'request_id': 'req-9a3313af-d69b-4272-a458-886ba03a4240'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:49.937496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3359528402bc4f7cb406f608c9982436', 'request_id': 'req-9a3313af-d69b-4272-a458-886ba03a4240'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:49.985904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] The token cache is being invalidate because project 3359528402bc4f7cb406f608c9982436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:49.986406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:49.987814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a3313af-d69b-4272-a458-886ba03a4240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:49.993330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 857/1720] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:49 2022] DELETE /identity/v3/projects/3359528402bc4f7cb406f608c9982436 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:51:49.995448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b0783-f275-4bca-b535-f097a69e9aa3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:49.995735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b0783-f275-4bca-b535-f097a69e9aa3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:49.996058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b0783-f275-4bca-b535-f097a69e9aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:50.033281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-7a2b0783-f275-4bca-b535-f097a69e9aa3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:50.036282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7a2b0783-f275-4bca-b535-f097a69e9aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:50.219056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:50.221157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:50.221651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:50.222087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] PATH_INFO: `/v3/projects/482abe2cc32c4191bd7d5012c3c5f9ac` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:50.224323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] RBAC: Authorizing `identity:delete_project(project_id=482abe2cc32c4191bd7d5012c3c5f9ac)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:50.228402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:50.270427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 858/1721] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:50.272681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076ec50-087a-4dc0-b0b0-e0b5164e48c5 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:50.272681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076ec50-087a-4dc0-b0b0-e0b5164e48c5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:50.272681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076ec50-087a-4dc0-b0b0-e0b5164e48c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:50.296925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2076ec50-087a-4dc0-b0b0-e0b5164e48c5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:50.298358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2076ec50-087a-4dc0-b0b0-e0b5164e48c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:50.338439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '482abe2cc32c4191bd7d5012c3c5f9ac', 'request_id': 'req-7da3ddfe-d485-4acd-a423-6ec5d59b37af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.354694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '482abe2cc32c4191bd7d5012c3c5f9ac', 'request_id': 'req-7da3ddfe-d485-4acd-a423-6ec5d59b37af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.373347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '482abe2cc32c4191bd7d5012c3c5f9ac', 'request_id': 'req-7da3ddfe-d485-4acd-a423-6ec5d59b37af'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.391705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:50.431204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] The token cache is being invalidate because project 482abe2cc32c4191bd7d5012c3c5f9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:50.431656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.432609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.434431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 864/1722] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:49 2022] DELETE /identity/v3/projects/482abe2cc32c4191bd7d5012c3c5f9ac => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:50.435549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:50.438450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:50.470552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 859/1723] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:51:50.471973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766cb99-27fe-44dc-aae7-17a7eabee2b2 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:50.472223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766cb99-27fe-44dc-aae7-17a7eabee2b2 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:50.472497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766cb99-27fe-44dc-aae7-17a7eabee2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:50.527583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3ddfe-d485-4acd-a423-6ec5d59b37af admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:50.540538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:50.541808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:50.542204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:50.542592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] PATH_INFO: `/v3/projects/74ca80207eda4415a3362e9caaed5317` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:50.542911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6766cb99-27fe-44dc-aae7-17a7eabee2b2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:50.546429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6766cb99-27fe-44dc-aae7-17a7eabee2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:50.551986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74ca80207eda4415a3362e9caaed5317)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:50.560540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:50.567637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:50.567841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 860/1724] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:50.572618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766cb99-27fe-44dc-aae7-17a7eabee2b2 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:50.597567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766cb99-27fe-44dc-aae7-17a7eabee2b2 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:50.609583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfc6a7-e59f-4a9f-a198-a1db28e1e480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:50.610621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfc6a7-e59f-4a9f-a198-a1db28e1e480 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:50.610908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfc6a7-e59f-4a9f-a198-a1db28e1e480 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:50.611177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfc6a7-e59f-4a9f-a198-a1db28e1e480 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:50.613766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfc6a7-e59f-4a9f-a198-a1db28e1e480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:50.614083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfc6a7-e59f-4a9f-a198-a1db28e1e480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:50.615703 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfc6a7-e59f-4a9f-a198-a1db28e1e480 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:50.672908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 861/1725] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:50.676468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c953c-5912-4f06-af94-12110081879b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:50.677075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c953c-5912-4f06-af94-12110081879b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:50.677662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c953c-5912-4f06-af94-12110081879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:50.696243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74ca80207eda4415a3362e9caaed5317', 'request_id': 'req-10a22eab-8a7a-4283-bf71-c2ba21e75213'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.711146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74ca80207eda4415a3362e9caaed5317', 'request_id': 'req-10a22eab-8a7a-4283-bf71-c2ba21e75213'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.719265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74ca80207eda4415a3362e9caaed5317', 'request_id': 'req-10a22eab-8a7a-4283-bf71-c2ba21e75213'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.744163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4b7c953c-5912-4f06-af94-12110081879b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:50.747737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4b7c953c-5912-4f06-af94-12110081879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:50.785848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] The token cache is being invalidate because project 74ca80207eda4415a3362e9caaed5317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:50.790966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.794704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:50.796632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 865/1726] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:50 2022] DELETE /identity/v3/projects/74ca80207eda4415a3362e9caaed5317 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:50.798180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:50.800581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:50.970987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 862/1727] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:50.972205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c953c-5912-4f06-af94-12110081879b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:50.982897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4b7c953c-5912-4f06-af94-12110081879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:51.031779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a22eab-8a7a-4283-bf71-c2ba21e75213 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:51.053449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437faa-56d6-4cfb-996c-69da6a487719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:51.053921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c953c-5912-4f06-af94-12110081879b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:51.054076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437faa-56d6-4cfb-996c-69da6a487719 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.054512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437faa-56d6-4cfb-996c-69da6a487719 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.054975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437faa-56d6-4cfb-996c-69da6a487719 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.055798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437faa-56d6-4cfb-996c-69da6a487719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:51.057894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437faa-56d6-4cfb-996c-69da6a487719 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:51.085807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:51.086656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.086930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.087193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] PATH_INFO: `/v3/users/b21bed13f7594ecbaf2194172abcfe8c` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.098072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21bed13f7594ecbaf2194172abcfe8c)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:51.116321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:51.159240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 866/1728] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:50 2022] POST /identity/v3/projects => generated 365 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:51.170148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.171180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.171630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.221421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:51.224091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:51.280714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21bed13f7594ecbaf2194172abcfe8c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:51.282017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21bed13f7594ecbaf2194172abcfe8c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:51.295194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21bed13f7594ecbaf2194172abcfe8c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:51.296247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21bed13f7594ecbaf2194172abcfe8c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:51.328793 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21bed13f7594ecbaf2194172abcfe8c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:51.379419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 863/1729] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:50 2022] DELETE /identity/v3/users/b21bed13f7594ecbaf2194172abcfe8c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:51.382078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:51.420623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 867/1730] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:51.421619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:51.424780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:51.427621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:51.443922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0faba53-7ac3-4a30-a38e-13339a994349 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:51.472059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:51.559196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff2def-08b6-413d-97bb-23ecedbdb5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:51.560065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff2def-08b6-413d-97bb-23ecedbdb5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.561156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff2def-08b6-413d-97bb-23ecedbdb5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.561573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff2def-08b6-413d-97bb-23ecedbdb5d2 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.562350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff2def-08b6-413d-97bb-23ecedbdb5d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:51.564544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff2def-08b6-413d-97bb-23ecedbdb5d2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:51.636656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:51.662146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 868/1731] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:51 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:51.665527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f4227-10f5-4293-9173-e6c195c354fd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.665841 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f4227-10f5-4293-9173-e6c195c354fd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.666685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f4227-10f5-4293-9173-e6c195c354fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.712432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b4048-98f8-42fe-a0e9-c5d61c9bdaa3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:51.725228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44367c-0377-4836-a258-6c8240426faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:51.727378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44367c-0377-4836-a258-6c8240426faa admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.727861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44367c-0377-4836-a258-6c8240426faa admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.728381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44367c-0377-4836-a258-6c8240426faa admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.729615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44367c-0377-4836-a258-6c8240426faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:51.730205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44367c-0377-4836-a258-6c8240426faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:51.733007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44367c-0377-4836-a258-6c8240426faa admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:51.751592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 864/1732] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:51.753949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8dcdb-c4df-4c6f-ad0c-f6d081db38cd None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.754775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8dcdb-c4df-4c6f-ad0c-f6d081db38cd None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.755245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8dcdb-c4df-4c6f-ad0c-f6d081db38cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.781229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-fa1f4227-10f5-4293-9173-e6c195c354fd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:51.782415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fa1f4227-10f5-4293-9173-e6c195c354fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:51.801721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-00d8dcdb-c4df-4c6f-ad0c-f6d081db38cd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:51.807111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-00d8dcdb-c4df-4c6f-ad0c-f6d081db38cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:51.814612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 869/1733] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:51.817043 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e913-dc0a-4e24-b532-3610a83ecf97 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.826732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e913-dc0a-4e24-b532-3610a83ecf97 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.827268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9e913-dc0a-4e24-b532-3610a83ecf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.834438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 865/1734] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:51.834438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8dcdb-c4df-4c6f-ad0c-f6d081db38cd None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:51.876878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8dcdb-c4df-4c6f-ad0c-f6d081db38cd None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:51.884768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-03b9e913-dc0a-4e24-b532-3610a83ecf97 None None] MFA Rules not processed for user `68288f4e2f57454fbb06eff0dd72cca3`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:51.886113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-03b9e913-dc0a-4e24-b532-3610a83ecf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:51.893396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac3363-0f8a-4ea9-9b96-efe886d0295a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:51.894529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac3363-0f8a-4ea9-9b96-efe886d0295a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.894819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac3363-0f8a-4ea9-9b96-efe886d0295a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.895406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac3363-0f8a-4ea9-9b96-efe886d0295a admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:51.897059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac3363-0f8a-4ea9-9b96-efe886d0295a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:51.902136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac3363-0f8a-4ea9-9b96-efe886d0295a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:51.923604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 866/1735] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:51.926044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8176d-0af0-4236-987d-ee637f804e43 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:51.926551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8176d-0af0-4236-987d-ee637f804e43 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:51.927058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8176d-0af0-4236-987d-ee637f804e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.043182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6fd8176d-0af0-4236-987d-ee637f804e43 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:52.045239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6fd8176d-0af0-4236-987d-ee637f804e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:52.065753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 870/1736] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:52.066685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9e913-dc0a-4e24-b532-3610a83ecf97 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.118543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9e913-dc0a-4e24-b532-3610a83ecf97 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:52.143963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:52.144612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:52.144891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:52.145160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.145814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:52.147726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:52.219788 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 867/1737] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:52.220720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8176d-0af0-4236-987d-ee637f804e43 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.229606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 871/1738] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:52 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:52.230806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.235405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:52.257682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8176d-0af0-4236-987d-ee637f804e43 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:52.364825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:52.365558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:52.365904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:52.366241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.366969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:52.367370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:52.369200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:52.386338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 868/1739] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:52.387222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.406831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8614-4454-469d-8d28-d40728273df3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:52.431364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:52.432492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:52.433177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:52.434813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.435992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:52.437986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:52.457999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f92ff-84ae-40ac-a2bc-834af8085412 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:52.479157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:52.480291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:52.480677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:52.480993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.484258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 869/1740] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:52.490420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.507851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:52.509718 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:52.536417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf10ed-ea99-4c30-ae0c-a6cfa6cc12be admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:52.550545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 872/1741] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:52.551744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.556155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:52.557219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:52.557766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:52.575050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.578238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:52.582431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:52.592668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779c8ef-e9b6-4b3e-9c96-f5077910d7cf service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:52.612771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:52.630880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:52.631507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:52.636071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:52.636400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.636985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 870/1742] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:52 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:52.637092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:52.637695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.638968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:52.655099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d415f-80d5-4c4e-99d0-5da3da90ba61 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:52.669923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:52.670612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:52.670888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:52.671156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] PATH_INFO: `/v3/projects/998ca17993014be79060dff0a1985d4b/users/e2a9a0f8da9b4e6fb825047e06188d75/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:52.712302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=998ca17993014be79060dff0a1985d4b, user_id=e2a9a0f8da9b4e6fb825047e06188d75, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:52.723159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:52.761446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 871/1743] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:52 2022] PUT /identity/v3/projects/998ca17993014be79060dff0a1985d4b/users/e2a9a0f8da9b4e6fb825047e06188d75/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:52.762365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.765238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:52.832003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 873/1744] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:52 2022] POST /identity/v3/projects => generated 387 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:52.849386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:52.882553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:53.156670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9413cc-c910-4e00-a818-10759b02d9af admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.164651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6fd01-13b6-42ab-af4c-03a1b7587214 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.175175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afff42-2d86-41f9-b0fb-a9e639369c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.176326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afff42-2d86-41f9-b0fb-a9e639369c1b admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.177283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afff42-2d86-41f9-b0fb-a9e639369c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.177466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afff42-2d86-41f9-b0fb-a9e639369c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.178132 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afff42-2d86-41f9-b0fb-a9e639369c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.180176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afff42-2d86-41f9-b0fb-a9e639369c1b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.196155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 874/1745] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:53.196155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7269-5051-44c2-b15a-5c22afc1e234 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.196155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7269-5051-44c2-b15a-5c22afc1e234 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.196488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7269-5051-44c2-b15a-5c22afc1e234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.249938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e80e7269-5051-44c2-b15a-5c22afc1e234 None None] MFA Rules not processed for user `e2a9a0f8da9b4e6fb825047e06188d75`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:53.252777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e80e7269-5051-44c2-b15a-5c22afc1e234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:53.291124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.291775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.292462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.292743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.415750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:53.422638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 875/1746] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:53.422638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7269-5051-44c2-b15a-5c22afc1e234 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:53.435960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.440443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.450494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7269-5051-44c2-b15a-5c22afc1e234 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.459684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 872/1747] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:51:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:51:53.460814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:53.478597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.479375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.482353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.484350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.491331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.492117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2f0ce-2b76-46d0-8a64-aa70f276711a service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.504218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.505402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.506154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.506678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.508024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.510179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.513962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.524364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 873/1748] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:53.525416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:53.537589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32792561-0ea5-45eb-bd0b-ad5a2cdcf626 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.547540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.548210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.548494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.548770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.552853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.557355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.601085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 876/1749] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:53 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:53.603118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:53.638897 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583a2f9-54e1-40f0-9af9-6d7ea426ac1f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.641350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 874/1750] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:53 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:53.643359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:53.818957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e5763-f669-4f7e-bf79-beb8b5fe3e62 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.839027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.839027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.839027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.839247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.839330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.839395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.839395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.839534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.839641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] PATH_INFO: `/v3/projects/24cd0d5ec1fc4c0dbb4420f67d56a481/users/97b6f597c6f043c69bc1e95bd741b945/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.840886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.882335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 875/1751] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:53.884809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:53.888547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=24cd0d5ec1fc4c0dbb4420f67d56a481, user_id=97b6f597c6f043c69bc1e95bd741b945, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.895379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.905095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:53.910343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b4b37-5c4c-4daf-8045-bdc87f13f8f1 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:53.923650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9beb21-21dc-4026-a0cd-b77e0d6d06c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:53.924378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9beb21-21dc-4026-a0cd-b77e0d6d06c1 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.924727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9beb21-21dc-4026-a0cd-b77e0d6d06c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.927218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9beb21-21dc-4026-a0cd-b77e0d6d06c1 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:53.929473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9beb21-21dc-4026-a0cd-b77e0d6d06c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:53.934569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9beb21-21dc-4026-a0cd-b77e0d6d06c1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:53.968797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 877/1752] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:53 2022] PUT /identity/v3/projects/24cd0d5ec1fc4c0dbb4420f67d56a481/users/97b6f597c6f043c69bc1e95bd741b945/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:53.968797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:53.995018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 876/1753] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:53 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:53.997277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0bda-584d-4534-afb4-8dcde09f930d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:53.997409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0bda-584d-4534-afb4-8dcde09f930d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:53.997602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0bda-584d-4534-afb4-8dcde09f930d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.098645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-3e2c0bda-584d-4534-afb4-8dcde09f930d None None] MFA Rules not processed for user `97b6f597c6f043c69bc1e95bd741b945`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:54.102654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3e2c0bda-584d-4534-afb4-8dcde09f930d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:54.275221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f7fc6-1372-4a3e-89df-69cbeafeae8f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.288627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.289150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.289441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.289749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.290696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.295940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:54.314046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 877/1754] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:54.318475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c0bda-584d-4534-afb4-8dcde09f930d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:54.323146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 878/1755] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:54.329255 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:54.330192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c0bda-584d-4534-afb4-8dcde09f930d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.335561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.336381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.336678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.336921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.337586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.340071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:54.343313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1434bc2-7571-45cd-a8bd-4af95534593a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.349780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.350446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.350730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.351041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.354878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 878/1756] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:54.356017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:54.359034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.361960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:54.375594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823916f-53c0-4da4-b6f8-78e94e6edf6b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.403270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.404772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.405874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.406338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] PATH_INFO: `/v3/projects/05d1f95ae31e4f1f9fd453c763e3405a/users/06dc554c514140469604a00671ea63cd/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.412918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 879/1757] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:54 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:54.413318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:54.447749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:54.456912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d1f95ae31e4f1f9fd453c763e3405a, user_id=06dc554c514140469604a00671ea63cd, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.470683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:54.510111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 879/1758] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:54 2022] PUT /identity/v3/projects/05d1f95ae31e4f1f9fd453c763e3405a/users/06dc554c514140469604a00671ea63cd/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:54.511911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:54.538287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c20bc5-2f15-4c45-ac76-9e85a7732560 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.645887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c27ca5-e885-44b7-9ea2-219ddd11544f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.646740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c27ca5-e885-44b7-9ea2-219ddd11544f admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.647106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c27ca5-e885-44b7-9ea2-219ddd11544f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.648804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c27ca5-e885-44b7-9ea2-219ddd11544f admin admin] PATH_INFO: `/v3/projects/d8a50dd8d9794dff96498f4329587367/users/4f46b81e9ae0498989091a5e86967eab/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.670851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9524c1-0e30-4d1c-891d-3bf6a6ca4733 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.683116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c27ca5-e885-44b7-9ea2-219ddd11544f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a50dd8d9794dff96498f4329587367, user_id=4f46b81e9ae0498989091a5e86967eab, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.689105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c27ca5-e885-44b7-9ea2-219ddd11544f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:54.697126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.699123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.700471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.700791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.701481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.703610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:54.712719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:54.725084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 880/1759] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:54 2022] PUT /identity/v3/projects/d8a50dd8d9794dff96498f4329587367/users/4f46b81e9ae0498989091a5e86967eab/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:54.728741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045152-ad1e-44ee-846a-98087e63864e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.731610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045152-ad1e-44ee-846a-98087e63864e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.732022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045152-ad1e-44ee-846a-98087e63864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.746733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 880/1760] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:54 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:54.748242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:54.769814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d1b33-c3ca-4377-b210-1a8ef717be7e admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.808096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1f045152-ad1e-44ee-846a-98087e63864e None None] MFA Rules not processed for user `06dc554c514140469604a00671ea63cd`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:54.809508 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1f045152-ad1e-44ee-846a-98087e63864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:54.888365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.888995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.889320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.889594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.890258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.897233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:54.928959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 881/1761] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:54.931344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:54.956211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21643af-3a62-4fc7-8432-156c6ca854e8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:54.970124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:54.970858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:54.971139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:54.971471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:54.972303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:54.974037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:55.008330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 882/1762] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:55.008330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:55.014419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 881/1763] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:55.015568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f045152-ad1e-44ee-846a-98087e63864e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:55.034573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79142-c840-4b29-8d1d-0ea9f855db08 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:55.052145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:55.052957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:55.053239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:55.053524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] PATH_INFO: `/v3/projects/d8a50dd8d9794dff96498f4329587367/users/4f46b81e9ae0498989091a5e86967eab/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:55.062303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a50dd8d9794dff96498f4329587367, user_id=4f46b81e9ae0498989091a5e86967eab, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:55.066308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:55.070162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f045152-ad1e-44ee-846a-98087e63864e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:55.095613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021669d9-188f-45e5-aea5-514179a0047a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:55.096349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021669d9-188f-45e5-aea5-514179a0047a admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:55.096631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021669d9-188f-45e5-aea5-514179a0047a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:55.096907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021669d9-188f-45e5-aea5-514179a0047a admin admin] PATH_INFO: `/v3/projects/aa0e6a567dc74bb9abf1535df4200921/users/7331982f6a044b0dad73dcfc62624690/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:55.131246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021669d9-188f-45e5-aea5-514179a0047a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0e6a567dc74bb9abf1535df4200921, user_id=7331982f6a044b0dad73dcfc62624690, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:55.139330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021669d9-188f-45e5-aea5-514179a0047a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:55.150134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 883/1764] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:55 2022] PUT /identity/v3/projects/d8a50dd8d9794dff96498f4329587367/users/4f46b81e9ae0498989091a5e86967eab/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:55.159046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:55.176260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 882/1765] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:55 2022] PUT /identity/v3/projects/aa0e6a567dc74bb9abf1535df4200921/users/7331982f6a044b0dad73dcfc62624690/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:55.185283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd71bc-c730-4a76-88ff-27d5fc1471b7 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:55.185631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd71bc-c730-4a76-88ff-27d5fc1471b7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:55.185909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd71bc-c730-4a76-88ff-27d5fc1471b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:55.325463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:55.355910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1960ba-fb70-4d9e-83ea-7f1260ac17fd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:55.365281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-6dfd71bc-c730-4a76-88ff-27d5fc1471b7 None None] MFA Rules not processed for user `4f46b81e9ae0498989091a5e86967eab`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:55.366545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6dfd71bc-c730-4a76-88ff-27d5fc1471b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:55.619616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d50e1-7064-4efc-91f8-b68b70885ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:55.620471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d50e1-7064-4efc-91f8-b68b70885ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:55.620820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d50e1-7064-4efc-91f8-b68b70885ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:55.621031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d50e1-7064-4efc-91f8-b68b70885ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:55.621966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d50e1-7064-4efc-91f8-b68b70885ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:55.625207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d50e1-7064-4efc-91f8-b68b70885ba8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:55.712555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 884/1766] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:55 2022] POST /identity/v3/projects => generated 387 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:55.714451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:55.714451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:55.714648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:55.742823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 883/1767] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:55.744501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd71bc-c730-4a76-88ff-27d5fc1471b7 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:55.768249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd71bc-c730-4a76-88ff-27d5fc1471b7 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:55.816119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] MFA Rules not processed for user `349ee49c297749468e09c4aa3dc9c218`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:55.817366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:55.893882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:55.937174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:55.938120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:55.938501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:55.939197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:55.940252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:55.942398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:55.984702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 884/1768] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:55.987427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:56.018706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a07d-0e5a-4a78-b52e-81614eaaede2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:56.038870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 885/1769] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Oct 4 08:51:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:56.040189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:56.042534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:56.049197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:56.051091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:56.053246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:56.053791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:56.057183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:56.059809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:56.138220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 885/1770] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:55 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:56.139097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:56.160015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeee285-9fda-4acd-8bb1-63f9a8bd722c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:56.167687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:56.169156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:56.169430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:56.169710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] PATH_INFO: `/v3/projects/aa0e6a567dc74bb9abf1535df4200921/users/7331982f6a044b0dad73dcfc62624690/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:56.174056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0e6a567dc74bb9abf1535df4200921, user_id=7331982f6a044b0dad73dcfc62624690, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:56.178447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:56.196513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 886/1771] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:56 2022] PUT /identity/v3/projects/aa0e6a567dc74bb9abf1535df4200921/users/7331982f6a044b0dad73dcfc62624690/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:56.197363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:56.206326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:56.250641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afff0c2-7f41-4058-8126-cff50a7112f1 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:56.438780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014e0e3-09f8-471b-81ab-4cf6a6fdd57c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:56.441584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:56.442544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:56.443107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:56.443953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:56.452863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:56.456405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:56.458145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:56.461984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:56.462804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] PATH_INFO: `/v3/projects/6d0f2821338a4628882790d108bfe02c` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:56.474870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0f2821338a4628882790d108bfe02c)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:56.486360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:56.611351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:56.613903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:56.662117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 886/1772] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:56.664623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:56.673285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0f2821338a4628882790d108bfe02c', 'request_id': 'req-916deaea-b372-4279-b30f-4d8b0ddd3694'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:56.689142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0f2821338a4628882790d108bfe02c', 'request_id': 'req-916deaea-b372-4279-b30f-4d8b0ddd3694'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:56.706595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0f2821338a4628882790d108bfe02c', 'request_id': 'req-916deaea-b372-4279-b30f-4d8b0ddd3694'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:56.754576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] The token cache is being invalidate because project 6d0f2821338a4628882790d108bfe02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:51:56.755224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:56.758848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:56.761312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 887/1773] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:51:56 2022] DELETE /identity/v3/projects/6d0f2821338a4628882790d108bfe02c => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:56.762313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:56.763991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:56.882934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a114447-a3b7-452d-be70-e7b44bb8e69b service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:56.895227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:56.929436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916deaea-b372-4279-b30f-4d8b0ddd3694 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:56.940836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:56.941730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:56.942047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:56.942395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:56.945321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:56.990273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:56.992420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:57.015229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 888/1774] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:57.016315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:57.018427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:57.044610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824806e-5f6c-4b3a-a13d-8b62642b07a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:57.045242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824806e-5f6c-4b3a-a13d-8b62642b07a3 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.045500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824806e-5f6c-4b3a-a13d-8b62642b07a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.045751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824806e-5f6c-4b3a-a13d-8b62642b07a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.065685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824806e-5f6c-4b3a-a13d-8b62642b07a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:57.069536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824806e-5f6c-4b3a-a13d-8b62642b07a3 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:57.095412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 887/1775] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:57.096771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.097051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.097423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.163465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] MFA Rules not processed for user `7331982f6a044b0dad73dcfc62624690`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:57.164762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:57.171999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8bdd6-25f0-4dcf-a61e-0e6fdf7bb251 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:57.195913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:57.196644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.196997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.197307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.197970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:57.199677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:57.244831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 889/1776] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:57.246165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:57.267741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d1bab-304e-4078-aa42-2022cce6d8fa admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:57.280967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0eeeb0-b3da-4127-835a-5bb149370e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:57.282122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0eeeb0-b3da-4127-835a-5bb149370e3e service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.282610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0eeeb0-b3da-4127-835a-5bb149370e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.283197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0eeeb0-b3da-4127-835a-5bb149370e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.303255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:57.303414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0eeeb0-b3da-4127-835a-5bb149370e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:57.306328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0eeeb0-b3da-4127-835a-5bb149370e3e service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:57.322363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 890/1777] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:57.337856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807038c-6f04-46c4-ab2f-c317db4dc5f6 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.338157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807038c-6f04-46c4-ab2f-c317db4dc5f6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.338462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807038c-6f04-46c4-ab2f-c317db4dc5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.387621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 888/1778] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:57.388767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:57.390946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:57.414009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1807038c-6f04-46c4-ab2f-c317db4dc5f6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:57.419549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1807038c-6f04-46c4-ab2f-c317db4dc5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:57.448797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 891/1779] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:57.450088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807038c-6f04-46c4-ab2f-c317db4dc5f6 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:57.463568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807038c-6f04-46c4-ab2f-c317db4dc5f6 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:57.471034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:57.471754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.472097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.472366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] PATH_INFO: `/v3/projects/02a040bd1ff5407d8cd6507a0286fcdd/users/0d6b4159a3b74e22a4536a6e64e44b47/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.491506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a040bd1ff5407d8cd6507a0286fcdd, user_id=0d6b4159a3b74e22a4536a6e64e44b47, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:57.495481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:57.520621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 892/1780] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:57 2022] PUT /identity/v3/projects/02a040bd1ff5407d8cd6507a0286fcdd/users/0d6b4159a3b74e22a4536a6e64e44b47/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:57.521645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:57.526455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:57.558280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4967d3a-18ed-4a88-aa60-67360d42518d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:57.631787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c96c9-62da-49df-9729-9526d05c2f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:57.632440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c96c9-62da-49df-9729-9526d05c2f41 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.633505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c96c9-62da-49df-9729-9526d05c2f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.633505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c96c9-62da-49df-9729-9526d05c2f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.635572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3a3c96c9-62da-49df-9729-9526d05c2f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:57.709512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42904c2-8368-410b-bb27-c0f11a12fa7b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:57.723443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:57.724183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.724455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.724902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.733998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c96c9-62da-49df-9729-9526d05c2f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:57.735622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c96c9-62da-49df-9729-9526d05c2f41 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:57.758753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 889/1781] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:57.762639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8afc0-a046-4095-bb38-4f07e7ad86b5 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:57.766247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8afc0-a046-4095-bb38-4f07e7ad86b5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:57.766847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8afc0-a046-4095-bb38-4f07e7ad86b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:57.831430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-3bf8afc0-a046-4095-bb38-4f07e7ad86b5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:57.834337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3bf8afc0-a046-4095-bb38-4f07e7ad86b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:57.866562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:57.886027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:57.887677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:57.932004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 893/1782] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:57.933591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.056966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:58.100449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 890/1783] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:58.109047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8afc0-a046-4095-bb38-4f07e7ad86b5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.129621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8afc0-a046-4095-bb38-4f07e7ad86b5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.159334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.160223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.161747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.162217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.163414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:51:58.164097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:58.178536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:58.189301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec8f8-da61-4c56-9d9f-3cd49a710d52 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.202802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.220921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.221342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.221712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.251390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 891/1784] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:51:58.252858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:58.256211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.258379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:58.284081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 894/1785] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:58.290768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.304978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c3d2b-7f1b-43f2-aa3d-3b64a8b0854c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.328264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.328996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.329465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.329580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542fa1f-0f09-4788-8c24-9421c3adebaa admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.329743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.331663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:58.342392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:58.344568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aec913-66b0-4235-b19a-de48541c4411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.346002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec913-66b0-4235-b19a-de48541c4411 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.346597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec913-66b0-4235-b19a-de48541c4411 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.347019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec913-66b0-4235-b19a-de48541c4411 admin admin] PATH_INFO: `/v3/projects/02a040bd1ff5407d8cd6507a0286fcdd/users/0d6b4159a3b74e22a4536a6e64e44b47/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.350074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aec913-66b0-4235-b19a-de48541c4411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a040bd1ff5407d8cd6507a0286fcdd, user_id=0d6b4159a3b74e22a4536a6e64e44b47, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:58.353008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aec913-66b0-4235-b19a-de48541c4411 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:58.434848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 892/1786] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:51:58 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:58.438854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.453706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 895/1787] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:58 2022] PUT /identity/v3/projects/02a040bd1ff5407d8cd6507a0286fcdd/users/0d6b4159a3b74e22a4536a6e64e44b47/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:58.469919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cf26e-994d-43b7-84ac-c5c4a729a080 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.470246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cf26e-994d-43b7-84ac-c5c4a729a080 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.470573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cf26e-994d-43b7-84ac-c5c4a729a080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.633761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-480cf26e-994d-43b7-84ac-c5c4a729a080 None None] MFA Rules not processed for user `0d6b4159a3b74e22a4536a6e64e44b47`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:58.635447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-480cf26e-994d-43b7-84ac-c5c4a729a080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:58.640457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc3eab-66fb-44c0-b232-99a57d3cabae admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.673141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.677525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.677827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.678107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.681505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:58.691429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:58.757220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 893/1788] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:51:58 2022] POST /identity/v3/users => generated 283 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:51:58.770005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.795715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271843-1552-4aa5-9a21-6ed481f67612 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.807412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 896/1789] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:51:58.810599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.811351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.811758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.812222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.813253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:58.815167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:58.830781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 894/1790] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:58.835349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cf26e-994d-43b7-84ac-c5c4a729a080 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.851419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cf26e-994d-43b7-84ac-c5c4a729a080 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.858741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.859817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.860439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.860848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.862032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:58.864315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:58.895232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 897/1791] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:51:58.896487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.902907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:58.929052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cd758-4f16-434e-82a9-f5e986d86c06 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:58.952409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bf02f-e567-4e8b-994b-d341fc26afbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:58.953104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bf02f-e567-4e8b-994b-d341fc26afbb admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:58.953385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bf02f-e567-4e8b-994b-d341fc26afbb admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:58.953661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bf02f-e567-4e8b-994b-d341fc26afbb admin admin] PATH_INFO: `/v3/projects/9a82c282db504701a7ddd68a8bdd0fa3/users/de4526e7c3ea494a89f05ebf5f41da78/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:58.992743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bf02f-e567-4e8b-994b-d341fc26afbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a82c282db504701a7ddd68a8bdd0fa3, user_id=de4526e7c3ea494a89f05ebf5f41da78, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:59.005876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bf02f-e567-4e8b-994b-d341fc26afbb admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:59.052545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 898/1792] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:51:58 2022] PUT /identity/v3/projects/9a82c282db504701a7ddd68a8bdd0fa3/users/de4526e7c3ea494a89f05ebf5f41da78/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:59.056620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec09c8e-aa1c-4f49-8596-a56d6466aada None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:59.056915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec09c8e-aa1c-4f49-8596-a56d6466aada None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:59.057185 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec09c8e-aa1c-4f49-8596-a56d6466aada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:59.122619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-6ec09c8e-aa1c-4f49-8596-a56d6466aada None None] MFA Rules not processed for user `de4526e7c3ea494a89f05ebf5f41da78`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:59.124026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6ec09c8e-aa1c-4f49-8596-a56d6466aada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:59.150413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:59.173649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1680c8-c708-4638-99f2-4b2d551401e9 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:59.322515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 899/1793] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:59.364146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:59.365253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:59.365709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:59.366128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:59.377997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:59.528905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec09c8e-aa1c-4f49-8596-a56d6466aada None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:59.534635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:59.536344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:59.546513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec09c8e-aa1c-4f49-8596-a56d6466aada None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:59.555843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 895/1794] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:51:59.565477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9bf02-6d9e-45ab-856e-44d16f0391fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:59.566126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9bf02-6d9e-45ab-856e-44d16f0391fd service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:59.566406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9bf02-6d9e-45ab-856e-44d16f0391fd service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:59.566679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9bf02-6d9e-45ab-856e-44d16f0391fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:59.604615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9bf02-6d9e-45ab-856e-44d16f0391fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:59.609858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9bf02-6d9e-45ab-856e-44d16f0391fd service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:59.622078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:59.639934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 900/1795] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:59.763724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738b8a6-25e1-4d59-b204-4a078f02449e service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:59.777045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:59.778112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:59.778602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:59.778880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] PATH_INFO: `/v3/users/06dc554c514140469604a00671ea63cd` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:59.781962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06dc554c514140469604a00671ea63cd)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:59.784228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:59.898590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06dc554c514140469604a00671ea63cd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:59.899740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06dc554c514140469604a00671ea63cd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:59.916397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06dc554c514140469604a00671ea63cd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:59.924371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06dc554c514140469604a00671ea63cd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:59.961965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06dc554c514140469604a00671ea63cd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:51:59.991186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 896/1796] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:51:59 2022] DELETE /identity/v3/users/06dc554c514140469604a00671ea63cd => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:51:59.996888 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:59.997415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:59.997415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:00.023949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:00.025806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:00.189902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 901/1797] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:00.250775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:00.253067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:00.313696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:00.362022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d44f7-db29-4506-b96a-a1c639e9bd15 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:00.368854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f93cc3-f0f1-4ed4-a0bd-1b9b70b868f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:00.369870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f93cc3-f0f1-4ed4-a0bd-1b9b70b868f3 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:00.370649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f93cc3-f0f1-4ed4-a0bd-1b9b70b868f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:00.371103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f93cc3-f0f1-4ed4-a0bd-1b9b70b868f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:00.395468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f93cc3-f0f1-4ed4-a0bd-1b9b70b868f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:00.397240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f93cc3-f0f1-4ed4-a0bd-1b9b70b868f3 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:00.419285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 897/1798] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:00.517060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:00.528426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:00.541020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a898-eaa7-4ce5-aa0d-7faae9aa5f41 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:00.550437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:00.552127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:00.552805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:00.553450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] PATH_INFO: `/v3/users/97b6f597c6f043c69bc1e95bd741b945` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:00.556745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b6f597c6f043c69bc1e95bd741b945)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:00.559724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:00.783923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b6f597c6f043c69bc1e95bd741b945'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:00.786778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b6f597c6f043c69bc1e95bd741b945'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:00.790726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b6f597c6f043c69bc1e95bd741b945'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:00.827320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b6f597c6f043c69bc1e95bd741b945'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:00.892740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-47a2adc9-cc23-4aa6-9fda-4c933839c27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b6f597c6f043c69bc1e95bd741b945'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:00.933667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 902/1799] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:00 2022] DELETE /identity/v3/users/97b6f597c6f043c69bc1e95bd741b945 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:00.937763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98914f66-e2fb-4777-9e5e-3635727442fd None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:00.938246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98914f66-e2fb-4777-9e5e-3635727442fd None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:00.938596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98914f66-e2fb-4777-9e5e-3635727442fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:00.998461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-98914f66-e2fb-4777-9e5e-3635727442fd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:01.002459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-98914f66-e2fb-4777-9e5e-3635727442fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:01.100320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff3d2b-62c1-4f01-bb81-2c80e6a803c7 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:01.101239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff3d2b-62c1-4f01-bb81-2c80e6a803c7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:01.101822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff3d2b-62c1-4f01-bb81-2c80e6a803c7 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:01.103495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 903/1800] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:52:01.129571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 898/1801] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:01.184767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff3d2b-62c1-4f01-bb81-2c80e6a803c7 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:01.186765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-99ff3d2b-62c1-4f01-bb81-2c80e6a803c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:01.213869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:01.317275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff3d2b-62c1-4f01-bb81-2c80e6a803c7 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:01.325762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397fca4-ff10-405b-9b7b-bbaf59c883cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:01.326017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397fca4-ff10-405b-9b7b-bbaf59c883cf service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:01.326280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397fca4-ff10-405b-9b7b-bbaf59c883cf service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:01.326609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397fca4-ff10-405b-9b7b-bbaf59c883cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:01.355173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397fca4-ff10-405b-9b7b-bbaf59c883cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:01.361091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397fca4-ff10-405b-9b7b-bbaf59c883cf service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:01.400204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 904/1802] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:01.480646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:01.481533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:01.482281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:01.525960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:01.528074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:01.532867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a58cb-dc19-4845-a4ac-4c6edea3ddfd None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:01.533397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a58cb-dc19-4845-a4ac-4c6edea3ddfd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:01.533838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a58cb-dc19-4845-a4ac-4c6edea3ddfd None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:01.535479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 905/1803] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:52:01.677238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:01.677547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:01.677983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:01.740222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:01.740222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:01.805863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 899/1804] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:52:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:01.956955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:01.959379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:02.023053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423442a0-c50d-4f6b-acba-78b337856c96 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:02.046391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:02.052017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:02.053736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:02.054517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 906/1805] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:52:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:02.055214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:02.099981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:02.101875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:02.111068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:02.113086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:02.155398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ef8d2-cb85-4cb8-9555-6e83973eddaf None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:02.156168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 900/1806] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:02.173983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:02.174574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:02.174855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:02.175117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:02.210130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:02.215792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:02.252555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 907/1807] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:02.585084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:02.586690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:02.624680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa15d1-b1a7-4910-842a-55d98cb5c5a0 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:02.653696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:02.654413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:02.654687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:02.654956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] PATH_INFO: `/v3/projects/05d1f95ae31e4f1f9fd453c763e3405a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:02.657570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d1f95ae31e4f1f9fd453c763e3405a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:02.659553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:02.813878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:02.828247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d1f95ae31e4f1f9fd453c763e3405a', 'request_id': 'req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:02.839946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d1f95ae31e4f1f9fd453c763e3405a', 'request_id': 'req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:02.852517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d1f95ae31e4f1f9fd453c763e3405a', 'request_id': 'req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:02.882333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] The token cache is being invalidate because project 05d1f95ae31e4f1f9fd453c763e3405a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:02.882913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:02.884140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:02.892138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 901/1808] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:02 2022] DELETE /identity/v3/projects/05d1f95ae31e4f1f9fd453c763e3405a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:02.892740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:02.926095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9cdfa-4f9e-4ad9-ab6a-8b4614b89b1b service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:03.190190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:03.190851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:03.190851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:03.191270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] PATH_INFO: `/v3/users/0d6b4159a3b74e22a4536a6e64e44b47` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:03.193105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6b4159a3b74e22a4536a6e64e44b47)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:03.195193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:03.273283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:03.323541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6b4159a3b74e22a4536a6e64e44b47'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.324829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6b4159a3b74e22a4536a6e64e44b47'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.326635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6b4159a3b74e22a4536a6e64e44b47'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.344530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6b4159a3b74e22a4536a6e64e44b47'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.380930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6b4159a3b74e22a4536a6e64e44b47'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.415428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 908/1809] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:02 2022] DELETE /identity/v3/users/0d6b4159a3b74e22a4536a6e64e44b47 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:03.503133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:03.504786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:03.538404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e98191-8395-4c7f-a84d-0ff429f2bc7b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:03.710865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:03.711490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:03.711490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:03.711490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] PATH_INFO: `/v3/projects/24cd0d5ec1fc4c0dbb4420f67d56a481` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:03.712948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24cd0d5ec1fc4c0dbb4420f67d56a481)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:03.715013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:03.855592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24cd0d5ec1fc4c0dbb4420f67d56a481', 'request_id': 'req-8282b236-7760-49d6-a138-ea9ea57fd3f4'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.869571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24cd0d5ec1fc4c0dbb4420f67d56a481', 'request_id': 'req-8282b236-7760-49d6-a138-ea9ea57fd3f4'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.891090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24cd0d5ec1fc4c0dbb4420f67d56a481', 'request_id': 'req-8282b236-7760-49d6-a138-ea9ea57fd3f4'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.952140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] The token cache is being invalidate because project 24cd0d5ec1fc4c0dbb4420f67d56a481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:03.952846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.955574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:03.959419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 902/1810] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:03 2022] DELETE /identity/v3/projects/24cd0d5ec1fc4c0dbb4420f67d56a481 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:03.963955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:03.965611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:03.985965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a22df-9b56-4a7e-ba42-b516d1edea32 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:04.204065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:04.204807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.204998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.205262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] PATH_INFO: `/v3/users/7331982f6a044b0dad73dcfc62624690` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.212127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7331982f6a044b0dad73dcfc62624690)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:04.214276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:04.288334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:04.291854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:04.324870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282b236-7760-49d6-a138-ea9ea57fd3f4 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:04.346686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:04.347435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.349708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.349708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] PATH_INFO: `/v3/projects/02a040bd1ff5407d8cd6507a0286fcdd` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.354857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a040bd1ff5407d8cd6507a0286fcdd)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:04.369564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:04.389342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7331982f6a044b0dad73dcfc62624690'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.399041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7331982f6a044b0dad73dcfc62624690'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.400713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7331982f6a044b0dad73dcfc62624690'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.414218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7331982f6a044b0dad73dcfc62624690'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.428292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3f45daf9-c5a8-4259-8d82-4195bd3bab52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7331982f6a044b0dad73dcfc62624690'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.463411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a040bd1ff5407d8cd6507a0286fcdd', 'request_id': 'req-48101df1-4c4c-4070-b7cd-a61c617c18c2'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.470451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a040bd1ff5407d8cd6507a0286fcdd', 'request_id': 'req-48101df1-4c4c-4070-b7cd-a61c617c18c2'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.471575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:04.479074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 909/1811] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:03 2022] DELETE /identity/v3/users/7331982f6a044b0dad73dcfc62624690 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:04.480454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a040bd1ff5407d8cd6507a0286fcdd', 'request_id': 'req-48101df1-4c4c-4070-b7cd-a61c617c18c2'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.513978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] The token cache is being invalidate because project 02a040bd1ff5407d8cd6507a0286fcdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:04.514422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.515216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-48101df1-4c4c-4070-b7cd-a61c617c18c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:04.517333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 903/1812] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:04 2022] DELETE /identity/v3/projects/02a040bd1ff5407d8cd6507a0286fcdd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:04.522415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea24f2-5aed-47de-9597-0616ac4219ac None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.522942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea24f2-5aed-47de-9597-0616ac4219ac None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.523203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea24f2-5aed-47de-9597-0616ac4219ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.552421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-92ea24f2-5aed-47de-9597-0616ac4219ac None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:04.554072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-92ea24f2-5aed-47de-9597-0616ac4219ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:04.611692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:04.738190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 910/1813] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:04.742937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b324b8-d9fc-4119-a460-007747bed6d4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.743546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b324b8-d9fc-4119-a460-007747bed6d4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.743853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b324b8-d9fc-4119-a460-007747bed6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.774137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c3b324b8-d9fc-4119-a460-007747bed6d4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:04.775378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c3b324b8-d9fc-4119-a460-007747bed6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:04.783635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 904/1814] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:04.788450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea24f2-5aed-47de-9597-0616ac4219ac None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:04.798535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea24f2-5aed-47de-9597-0616ac4219ac None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:04.803500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:04.804905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.804905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.804905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.805562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:04.806316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:04.807475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:04.824585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 911/1815] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:04.825578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b324b8-d9fc-4119-a460-007747bed6d4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:04.835170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b324b8-d9fc-4119-a460-007747bed6d4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:04.840236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:04.840858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.841196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.841393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.842553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:04.844362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:04.877128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 905/1816] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:04 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:04.881737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:04.892374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:04.894523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:04.903103 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c4a7f-4403-4815-9115-3d190601ab3e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:04.974555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:04.975183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.975460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.975730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.977150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:04.979069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:04.979442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464c9ae-fb59-436c-8b2d-5018721ff9ee admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:04.984210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:04.984872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:04.985190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:04.985464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] PATH_INFO: `/v3/projects/aa0e6a567dc74bb9abf1535df4200921` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:04.986567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0e6a567dc74bb9abf1535df4200921)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:04.989966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.001833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 912/1817] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:04 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:05.006006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.015532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590689b4-fc2a-444a-a054-baffad0653cd admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.020702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.021288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.021560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.021826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.022462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.024110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.034555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 913/1818] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:05.038474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.038659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0e6a567dc74bb9abf1535df4200921', 'request_id': 'req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:05.045488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0e6a567dc74bb9abf1535df4200921', 'request_id': 'req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:05.048539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f06e1-9096-4053-a8ee-574006e0bd9c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.051208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0e6a567dc74bb9abf1535df4200921', 'request_id': 'req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:05.066755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] The token cache is being invalidate because project aa0e6a567dc74bb9abf1535df4200921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:05.067122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:05.067987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2c8277d6-12a0-43f9-b9b3-6c754b0ac0ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:05.069797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 906/1819] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:04 2022] DELETE /identity/v3/projects/aa0e6a567dc74bb9abf1535df4200921 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:05.074334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455e49-456c-4dc4-b2b9-994a9330d6a7 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.074614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455e49-456c-4dc4-b2b9-994a9330d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.074896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455e49-456c-4dc4-b2b9-994a9330d6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.096653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-65455e49-456c-4dc4-b2b9-994a9330d6a7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:05.097873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-65455e49-456c-4dc4-b2b9-994a9330d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.110502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.111126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.111412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.111677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.112775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.114363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.124681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 914/1820] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:05.128758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.130713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.143036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53187-af08-43e6-af75-45f89a69348e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.144482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:05.148565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c631cb-a792-4621-9326-8638c18ca04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.149375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c631cb-a792-4621-9326-8638c18ca04b admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.149620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c631cb-a792-4621-9326-8638c18ca04b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.149929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c631cb-a792-4621-9326-8638c18ca04b admin admin] PATH_INFO: `/v3/projects/cca7a41e950c4a6e821596587045d046/users/6652e56dd3274b2fb481f5c6e22c3723/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.165061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c631cb-a792-4621-9326-8638c18ca04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca7a41e950c4a6e821596587045d046, user_id=6652e56dd3274b2fb481f5c6e22c3723, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.165457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 907/1821] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:05.167841 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c631cb-a792-4621-9326-8638c18ca04b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.173626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd492e-695b-4640-957d-3c3a98533a05 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.173961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd492e-695b-4640-957d-3c3a98533a05 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.174216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd492e-695b-4640-957d-3c3a98533a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.182529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 915/1822] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:05 2022] PUT /identity/v3/projects/cca7a41e950c4a6e821596587045d046/users/6652e56dd3274b2fb481f5c6e22c3723/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:05.193211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4deffc-959e-4a30-a868-c1b03cc9f961 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.193495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4deffc-959e-4a30-a868-c1b03cc9f961 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.193752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4deffc-959e-4a30-a868-c1b03cc9f961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.202130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a4cd492e-695b-4640-957d-3c3a98533a05 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:05.204169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a4cd492e-695b-4640-957d-3c3a98533a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.214882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:05.225803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-7b4deffc-959e-4a30-a868-c1b03cc9f961 None None] MFA Rules not processed for user `6652e56dd3274b2fb481f5c6e22c3723`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:05.227801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7b4deffc-959e-4a30-a868-c1b03cc9f961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.278401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 908/1823] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:05.282494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd492e-695b-4640-957d-3c3a98533a05 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.291352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd492e-695b-4640-957d-3c3a98533a05 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.296057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.296677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.296958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.297218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.297860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:05.298153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.299769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.303265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 916/1824] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:05.304287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4deffc-959e-4a30-a868-c1b03cc9f961 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.316934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 909/1825] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:52:05.317079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4deffc-959e-4a30-a868-c1b03cc9f961 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.317840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.319454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.320083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.320404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.320672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.321324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.323239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.327255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36673317-7c41-40ce-863e-0febaa0d907a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.331716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.332326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.332960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.332960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.333502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.335353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.353459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 917/1826] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:05.358128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.365258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 910/1827] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:05.370159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.428516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2311e-5b50-45e7-a7a7-58f1b05bc0c8 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.459652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602745-0034-47ce-9fd6-d26bef8bc587 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.464567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.465204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.465468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.465737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.466400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.468388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.494501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 911/1828] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:05.497028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.498542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.500554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.500840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.501115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.501763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.503666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.508387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8779973-bdd0-41d3-be23-5a3260bda997 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.513129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.513739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.514025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.514300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.514948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.516579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.527458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 912/1829] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:05.531555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.537182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 918/1830] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:05.539281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.543244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb417def-c704-418a-a827-5c7f45e5d699 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.548810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.549430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.549739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.549836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe271519-e8d4-4578-b51a-bede2fde53eb admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.550008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.550652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.552284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.554747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.555376 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.555869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.556340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.557684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.558769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.564835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 913/1831] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:05.568433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.573507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 919/1832] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:05.574588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.580142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:05.584087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d92f49-a64b-4106-9e65-f36766a2c4ec admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.585852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8fbe-5455-4d23-ac78-23d3e1ce2cde admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.589144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.589825 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.590191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.590766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] PATH_INFO: `/v3/projects/c4f7007652c24550b98f5c85f9e8a0a6/users/e56394ef9f144ebf9e8b8bc3f758bacc/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.593174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08872f6e-69bf-40c8-b156-33b4d9bad3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.593842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872f6e-69bf-40c8-b156-33b4d9bad3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.594118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872f6e-69bf-40c8-b156-33b4d9bad3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.594580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872f6e-69bf-40c8-b156-33b4d9bad3c1 admin admin] PATH_INFO: `/v3/projects/1ef7232b3b9a459b83268b9f549baf76/users/7dd3d3c6ee96426692e27d1e48e181cc/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.608119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f7007652c24550b98f5c85f9e8a0a6, user_id=e56394ef9f144ebf9e8b8bc3f758bacc, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.610215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872f6e-69bf-40c8-b156-33b4d9bad3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef7232b3b9a459b83268b9f549baf76, user_id=7dd3d3c6ee96426692e27d1e48e181cc, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.610997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.613359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08872f6e-69bf-40c8-b156-33b4d9bad3c1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.614032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:05.627728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 914/1833] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:05 2022] PUT /identity/v3/projects/c4f7007652c24550b98f5c85f9e8a0a6/users/e56394ef9f144ebf9e8b8bc3f758bacc/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:05.628533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 920/1834] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:05 2022] PUT /identity/v3/projects/1ef7232b3b9a459b83268b9f549baf76/users/7dd3d3c6ee96426692e27d1e48e181cc/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:05.632496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.638225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.638510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.638824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.671629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] MFA Rules not processed for user `e56394ef9f144ebf9e8b8bc3f758bacc`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:05.672872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.702692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e673f6-a8ac-4628-8b54-b5c81581903f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.740060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 921/1835] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:05.757448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.761735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.773569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.775617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.775617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.775617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.776072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.778133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.790761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 915/1836] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:05.808018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.808018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6acc95-24d4-4dbb-b614-67d4987b09eb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.818753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62f039-ce33-4407-ba91-070e934e5b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.818753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62f039-ce33-4407-ba91-070e934e5b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.818753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62f039-ce33-4407-ba91-070e934e5b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.819102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62f039-ce33-4407-ba91-070e934e5b28 admin admin] PATH_INFO: `/v3/projects/1ef7232b3b9a459b83268b9f549baf76/users/7dd3d3c6ee96426692e27d1e48e181cc/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.821155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62f039-ce33-4407-ba91-070e934e5b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef7232b3b9a459b83268b9f549baf76, user_id=7dd3d3c6ee96426692e27d1e48e181cc, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.823802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62f039-ce33-4407-ba91-070e934e5b28 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.845389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 916/1837] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:05 2022] PUT /identity/v3/projects/1ef7232b3b9a459b83268b9f549baf76/users/7dd3d3c6ee96426692e27d1e48e181cc/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:05.847027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.847421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.847704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.860879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3810a9-e08c-4536-b5d9-258646030e99 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:05.884021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] MFA Rules not processed for user `7dd3d3c6ee96426692e27d1e48e181cc`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:05.885570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.914790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:05.915525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:05.916012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:05.916698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:05.937345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:05.967658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 917/1838] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:05.978988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:05.980423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:05.984386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:05.986012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:05.991806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:05.996934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 922/1839] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:52:06.049930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1c9fd-6e62-4e20-8e25-38501566b461 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:06.053833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dca2b-0a83-4124-b016-394fec186d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:06.054574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dca2b-0a83-4124-b016-394fec186d0b service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:06.054911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dca2b-0a83-4124-b016-394fec186d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:06.055472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dca2b-0a83-4124-b016-394fec186d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:06.065132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dca2b-0a83-4124-b016-394fec186d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:06.066772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dca2b-0a83-4124-b016-394fec186d0b service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:06.078199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 918/1840] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:52:06.187914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:06.199917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362112e6-745b-48ce-bb10-e9907767b581 service glance] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:06.204534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:06.205223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:06.205594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:06.205915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:06.267176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:06.268642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:06.278030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 923/1841] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:09.110713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbfcc6-f0c8-44f5-b572-8713507d2dc5 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:09.111258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbfcc6-f0c8-44f5-b572-8713507d2dc5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:09.111258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbfcc6-f0c8-44f5-b572-8713507d2dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:09.132485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b0fbfcc6-f0c8-44f5-b572-8713507d2dc5 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:09.133787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b0fbfcc6-f0c8-44f5-b572-8713507d2dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:09.197947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 919/1842] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:52:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:09.205020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:09.206544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:09.294040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6695d-d2b1-4ea3-9b86-570a27e3bb52 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:09.299182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4e8b6-c86c-45eb-8d82-6cdb193abfae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:09.299808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e8b6-c86c-45eb-8d82-6cdb193abfae service swift] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:09.300121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e8b6-c86c-45eb-8d82-6cdb193abfae service swift] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:09.300403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4e8b6-c86c-45eb-8d82-6cdb193abfae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:09.319219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4e8b6-c86c-45eb-8d82-6cdb193abfae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:09.321033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4e8b6-c86c-45eb-8d82-6cdb193abfae service swift] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:09.329548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 924/1843] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:52:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:52:12.412234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbfcc6-f0c8-44f5-b572-8713507d2dc5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:12.487874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbfcc6-f0c8-44f5-b572-8713507d2dc5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:12.493075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:12.493881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:12.494336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:12.494765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] PATH_INFO: `/v3/users/6652e56dd3274b2fb481f5c6e22c3723` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:12.496058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6652e56dd3274b2fb481f5c6e22c3723)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:12.506673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:12.567172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6652e56dd3274b2fb481f5c6e22c3723'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:12.567833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6652e56dd3274b2fb481f5c6e22c3723'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:12.577640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6652e56dd3274b2fb481f5c6e22c3723'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:12.578852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6652e56dd3274b2fb481f5c6e22c3723'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:12.581152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:12.603521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6652e56dd3274b2fb481f5c6e22c3723'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:12.623847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 920/1844] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:12 2022] DELETE /identity/v3/users/6652e56dd3274b2fb481f5c6e22c3723 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:12.628160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:12.628780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:12.628780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:12.651371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:12.653802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:12.710379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:12.729556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 925/1845] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:12.739659 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:12.741402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:12.801864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff6db5-8348-4f53-bc45-66545fa85c7d admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:12.806322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:12.807003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:12.807354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:12.807607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:12.826879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:12.828819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:12.841508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 921/1846] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:13.215476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:13.217220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:13.233365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af87c95-44f9-4000-9fc9-385f533f818d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:13.238997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:13.239934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:13.240387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:13.240826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] PATH_INFO: `/v3/projects/cca7a41e950c4a6e821596587045d046` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:13.242176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca7a41e950c4a6e821596587045d046)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:13.244354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:13.303305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca7a41e950c4a6e821596587045d046', 'request_id': 'req-faada607-318f-48a9-ad44-cfa54bddae0c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.310054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca7a41e950c4a6e821596587045d046', 'request_id': 'req-faada607-318f-48a9-ad44-cfa54bddae0c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.316269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca7a41e950c4a6e821596587045d046', 'request_id': 'req-faada607-318f-48a9-ad44-cfa54bddae0c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.331939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] The token cache is being invalidate because project cca7a41e950c4a6e821596587045d046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:13.332458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.333288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.337404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 926/1847] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:13 2022] DELETE /identity/v3/projects/cca7a41e950c4a6e821596587045d046 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:13.341586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:13.351213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911147b8-1703-45aa-903c-6b8ea42f3048 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:13.414814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:13.415521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:13.415801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:13.416100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] PATH_INFO: `/v3/users/7dd3d3c6ee96426692e27d1e48e181cc` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:13.417230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd3d3c6ee96426692e27d1e48e181cc)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:13.419135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:13.474993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd3d3c6ee96426692e27d1e48e181cc'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.475572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd3d3c6ee96426692e27d1e48e181cc'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.484033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd3d3c6ee96426692e27d1e48e181cc'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.484867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd3d3c6ee96426692e27d1e48e181cc'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.503246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e9fe36dc-6ee8-451d-a8de-ae679fa0a21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd3d3c6ee96426692e27d1e48e181cc'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:13.513718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:13.519454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 922/1848] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:13 2022] DELETE /identity/v3/users/7dd3d3c6ee96426692e27d1e48e181cc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:13.860309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:13.862513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:13.892464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faada607-318f-48a9-ad44-cfa54bddae0c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:13.956403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:13.957126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:13.957490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:13.957809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] PATH_INFO: `/v3/projects/1ef7232b3b9a459b83268b9f549baf76` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:13.959003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef7232b3b9a459b83268b9f549baf76)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:13.961121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.032681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef7232b3b9a459b83268b9f549baf76', 'request_id': 'req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:14.040292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef7232b3b9a459b83268b9f549baf76', 'request_id': 'req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:14.047138 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef7232b3b9a459b83268b9f549baf76', 'request_id': 'req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:14.059367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:14.066242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] The token cache is being invalidate because project 1ef7232b3b9a459b83268b9f549baf76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:14.067232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:14.068597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9e24838b-1aed-42a4-b053-6d8cdb5aaed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:14.072248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 927/1849] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:13 2022] DELETE /identity/v3/projects/1ef7232b3b9a459b83268b9f549baf76 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:14.077063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393e48-e242-432a-9518-3e5e369a69df None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.077345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393e48-e242-432a-9518-3e5e369a69df None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.077618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393e48-e242-432a-9518-3e5e369a69df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.100439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-58393e48-e242-432a-9518-3e5e369a69df None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:14.101741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-58393e48-e242-432a-9518-3e5e369a69df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:14.166950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 923/1850] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:14.175132 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f66aa-8bb5-4c1e-a578-9598d59518cf None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.175415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f66aa-8bb5-4c1e-a578-9598d59518cf None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.175692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f66aa-8bb5-4c1e-a578-9598d59518cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.197490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-8c4f66aa-8bb5-4c1e-a578-9598d59518cf None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:14.199146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8c4f66aa-8bb5-4c1e-a578-9598d59518cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:14.208917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 928/1851] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:14.213373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58393e48-e242-432a-9518-3e5e369a69df None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.225508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58393e48-e242-432a-9518-3e5e369a69df None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.233110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.233871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.234215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.234482 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.235140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:14.236018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.237470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.253823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 924/1852] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:14.255082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f66aa-8bb5-4c1e-a578-9598d59518cf None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.265519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f66aa-8bb5-4c1e-a578-9598d59518cf None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.272887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.273658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.273956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.274303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.274957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.277148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.315954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 929/1853] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:14 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:14.317178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.327572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f7f30-026e-4c45-a107-fb80c2693165 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.388134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.388758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.389054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.389307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.389941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.391931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.422937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 925/1854] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:14 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:14.427302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.440128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97ff5e-26c1-4225-ab50-644066e02de2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.444908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.445498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.445778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.446049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.446697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.448311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.462378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 930/1855] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:14.463673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.473727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c998fb-c3ba-4092-81ed-af7444369597 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.478488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e46b7a-9de3-43fd-b081-09fe4ca75142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.479132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46b7a-9de3-43fd-b081-09fe4ca75142 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.479379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46b7a-9de3-43fd-b081-09fe4ca75142 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.479650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46b7a-9de3-43fd-b081-09fe4ca75142 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.480419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e46b7a-9de3-43fd-b081-09fe4ca75142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.482025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e46b7a-9de3-43fd-b081-09fe4ca75142 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.493006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 926/1856] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:14.496995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.507564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cddef-6b4b-48f6-a7ab-f1a073a52748 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.512524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.513199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.513478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.513755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] PATH_INFO: `/v3/projects/75dd9786ee314fc6bcb709f7938a47bb/users/b1af4183914541bc973e55bf47590576/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.529746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=75dd9786ee314fc6bcb709f7938a47bb, user_id=b1af4183914541bc973e55bf47590576, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.533218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.549330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 931/1857] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:14 2022] PUT /identity/v3/projects/75dd9786ee314fc6bcb709f7938a47bb/users/b1af4183914541bc973e55bf47590576/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:14.559731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e172c44-d475-4fdd-84ed-92fa30916379 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.560759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e172c44-d475-4fdd-84ed-92fa30916379 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.561124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e172c44-d475-4fdd-84ed-92fa30916379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.599378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-7e172c44-d475-4fdd-84ed-92fa30916379 None None] MFA Rules not processed for user `b1af4183914541bc973e55bf47590576`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:14.600781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7e172c44-d475-4fdd-84ed-92fa30916379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:14.655848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:14.667710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 927/1858] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:14.672372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.737646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dd10e-c152-46b3-abfb-85fde661f9fb admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.743161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.743955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.744246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.744516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.745222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.747295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.823271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 932/1859] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:14 2022] POST /identity/v3/projects => generated 389 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:14.825385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e172c44-d475-4fdd-84ed-92fa30916379 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:14.840896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e172c44-d475-4fdd-84ed-92fa30916379 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:14.943780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:14.944981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:14.945101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:14.945502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:14.946162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:14.948401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:14.983225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 928/1860] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:14 2022] POST /identity/v3/users => generated 301 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:14.991962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.005002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7bcf-d191-4660-a3fe-82fced41fc42 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.010430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.011033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.011307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.011580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.012299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.014303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.027875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 933/1861] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:15.032020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.045815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2aaf1-60ee-46eb-9366-188e082d7187 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.051079 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.052678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.052678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.052888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] PATH_INFO: `/v3/projects/0092148a1a6148e9ad6beb868ad272df/users/54246dcdcf214e86937d36fb96e4f0c1/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.076049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0092148a1a6148e9ad6beb868ad272df, user_id=54246dcdcf214e86937d36fb96e4f0c1, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.079247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.101654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 929/1862] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:15 2022] PUT /identity/v3/projects/0092148a1a6148e9ad6beb868ad272df/users/54246dcdcf214e86937d36fb96e4f0c1/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:15.109142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.123000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db65328-0b10-4eb5-bc62-38669dd468cf admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.147132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.149197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:15.204897 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.205200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.205475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.205743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.206441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.208266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.223436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 934/1863] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:15.225058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.240114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c493c6c-5364-49e7-976b-b9f710d9a14a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.241794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb16c3-8b80-4b1b-8d74-4a3138d98ab5 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.245781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed3fd3-c909-4ebb-a08d-b3932c34fc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.246460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3fd3-c909-4ebb-a08d-b3932c34fc60 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.246746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3fd3-c909-4ebb-a08d-b3932c34fc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.247023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3fd3-c909-4ebb-a08d-b3932c34fc60 admin admin] PATH_INFO: `/v3/projects/0092148a1a6148e9ad6beb868ad272df/users/54246dcdcf214e86937d36fb96e4f0c1/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.247146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.249061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed3fd3-c909-4ebb-a08d-b3932c34fc60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0092148a1a6148e9ad6beb868ad272df, user_id=54246dcdcf214e86937d36fb96e4f0c1, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.251663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.252077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.252449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] PATH_INFO: `/v3/users/e2a9a0f8da9b4e6fb825047e06188d75` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.253855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a9a0f8da9b4e6fb825047e06188d75)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.255934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.267095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed3fd3-c909-4ebb-a08d-b3932c34fc60 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.270291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:15.288017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 935/1864] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:15 2022] PUT /identity/v3/projects/0092148a1a6148e9ad6beb868ad272df/users/54246dcdcf214e86937d36fb96e4f0c1/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:15.301593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.302120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.302415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.321183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a9a0f8da9b4e6fb825047e06188d75'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.321927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a9a0f8da9b4e6fb825047e06188d75'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.331215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a9a0f8da9b4e6fb825047e06188d75'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.332132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a9a0f8da9b4e6fb825047e06188d75'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.337260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:15.341450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] MFA Rules not processed for user `54246dcdcf214e86937d36fb96e4f0c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:15.343354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:15.350495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-45e4ab8b-d8ce-4fc3-8eed-387087fd1763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a9a0f8da9b4e6fb825047e06188d75'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.368986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 930/1865] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:15 2022] DELETE /identity/v3/users/e2a9a0f8da9b4e6fb825047e06188d75 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:15.373402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.373695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.374058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.397320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:15.398633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:15.460715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 936/1866] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:15.472744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.474243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:15.491795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 931/1867] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:15.502144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.503621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:15.541090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d99d2-5a76-433a-ab51-9489b95ec87c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.565864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e4945-ff53-4b14-a93f-9d724845821f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.582962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.583609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.583908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.584185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.594634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.596140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.597283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea02b43-24fa-41df-9608-41f801b294a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.597904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea02b43-24fa-41df-9608-41f801b294a5 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.598186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea02b43-24fa-41df-9608-41f801b294a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.598460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea02b43-24fa-41df-9608-41f801b294a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.607421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 932/1868] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:15.658037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:15.680820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea02b43-24fa-41df-9608-41f801b294a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.683474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea02b43-24fa-41df-9608-41f801b294a5 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.704003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 937/1869] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:52:15.848401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:15.849879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:15.872792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde621d4-7811-422c-8171-4c9a68d38af4 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:15.877801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:15.878882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:15.879642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:15.880324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] PATH_INFO: `/v3/users/b1af4183914541bc973e55bf47590576` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:15.882390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1af4183914541bc973e55bf47590576)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:15.884694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:15.964802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1af4183914541bc973e55bf47590576'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.966061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1af4183914541bc973e55bf47590576'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.978890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1af4183914541bc973e55bf47590576'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.979806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1af4183914541bc973e55bf47590576'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:15.997942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1af4183914541bc973e55bf47590576'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.003089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:16.016559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 933/1870] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:15 2022] DELETE /identity/v3/users/b1af4183914541bc973e55bf47590576 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:16.021152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:16.021638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:16.021638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:16.031577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:16.033188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.053559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:16.056998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.116332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c2696-2202-46ff-9f08-e6e6c417ffd2 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:16.124311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:16.125284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:16.125867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:16.128374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] PATH_INFO: `/v3/projects/998ca17993014be79060dff0a1985d4b` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:16.130339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=998ca17993014be79060dff0a1985d4b)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:16.132843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:16.156515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 938/1871] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:16.168379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:16.169962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.197742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998ca17993014be79060dff0a1985d4b', 'request_id': 'req-8faaa219-2852-41e0-acd4-7ef301174849'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.205473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998ca17993014be79060dff0a1985d4b', 'request_id': 'req-8faaa219-2852-41e0-acd4-7ef301174849'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.213477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998ca17993014be79060dff0a1985d4b', 'request_id': 'req-8faaa219-2852-41e0-acd4-7ef301174849'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.236748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] The token cache is being invalidate because project 998ca17993014be79060dff0a1985d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:16.236748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.239324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.242345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 934/1872] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:16 2022] DELETE /identity/v3/projects/998ca17993014be79060dff0a1985d4b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:16.247207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:16.249182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.249515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6ce80-606f-4359-810a-47825edaaf05 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:16.308966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:16.309787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:16.310128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:16.310721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:16.313040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.354569 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faaa219-2852-41e0-acd4-7ef301174849 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:16.374925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:16.381004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:16.381707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:16.382123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] PATH_INFO: `/v3/users/4f46b81e9ae0498989091a5e86967eab` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:16.384395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f46b81e9ae0498989091a5e86967eab)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:16.387468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:16.408901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:16.427553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:16.429538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:16.433152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:16.446790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 939/1873] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:52:16.476443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f46b81e9ae0498989091a5e86967eab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.477127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f46b81e9ae0498989091a5e86967eab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.489799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f46b81e9ae0498989091a5e86967eab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.492532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f46b81e9ae0498989091a5e86967eab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.525755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f46b81e9ae0498989091a5e86967eab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.542147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 935/1874] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:16 2022] DELETE /identity/v3/users/4f46b81e9ae0498989091a5e86967eab => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:16.772264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:16.774079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.790015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d41040-e837-47b5-9e68-0c3d1ffc7f26 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:16.885817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:16.886848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:16.887589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:16.887866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] PATH_INFO: `/v3/projects/75dd9786ee314fc6bcb709f7938a47bb` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:16.889223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75dd9786ee314fc6bcb709f7938a47bb)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:16.891557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:16.957966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75dd9786ee314fc6bcb709f7938a47bb', 'request_id': 'req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.972578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75dd9786ee314fc6bcb709f7938a47bb', 'request_id': 'req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:16.985591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:16.987024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.992223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75dd9786ee314fc6bcb709f7938a47bb', 'request_id': 'req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.001729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b5491-f13c-4491-8acc-ee0ebf51d5d5 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.007303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] The token cache is being invalidate because project 75dd9786ee314fc6bcb709f7938a47bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:17.007799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.008708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.011673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 940/1875] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:16 2022] DELETE /identity/v3/projects/75dd9786ee314fc6bcb709f7938a47bb => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:17.017095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.017709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.071440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.072076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.072380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.072652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] PATH_INFO: `/v3/projects/d8a50dd8d9794dff96498f4329587367` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.073761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a50dd8d9794dff96498f4329587367)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.075776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.093246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a935efc-4600-4b8e-a361-e4e71ea2e1b2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.100117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.100979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.101340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.101663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] PATH_INFO: `/v3/users/54246dcdcf214e86937d36fb96e4f0c1` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.102782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] RBAC: Authorizing `identity:delete_user(user_id=54246dcdcf214e86937d36fb96e4f0c1)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.104814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.124313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a50dd8d9794dff96498f4329587367', 'request_id': 'req-b2f01dd1-f7d2-4bc1-8649-6609600970d9'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.130083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:17.130865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a50dd8d9794dff96498f4329587367', 'request_id': 'req-b2f01dd1-f7d2-4bc1-8649-6609600970d9'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.137357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a50dd8d9794dff96498f4329587367', 'request_id': 'req-b2f01dd1-f7d2-4bc1-8649-6609600970d9'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.153836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] The token cache is being invalidate because project d8a50dd8d9794dff96498f4329587367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:17.154124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.154896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.157348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 936/1876] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:16 2022] DELETE /identity/v3/projects/d8a50dd8d9794dff96498f4329587367 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:17.157941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.161492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.165095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54246dcdcf214e86937d36fb96e4f0c1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.166066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54246dcdcf214e86937d36fb96e4f0c1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.166970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54246dcdcf214e86937d36fb96e4f0c1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.175509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54246dcdcf214e86937d36fb96e4f0c1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.195602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a4a858de-a971-4e80-9208-41035d2de0be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54246dcdcf214e86937d36fb96e4f0c1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.209821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:17.212721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 941/1877] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:17 2022] DELETE /identity/v3/users/54246dcdcf214e86937d36fb96e4f0c1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:17.213963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736e36-87d2-40fe-982a-005f44f5e444 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.214297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736e36-87d2-40fe-982a-005f44f5e444 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.214569 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5736e36-87d2-40fe-982a-005f44f5e444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.239241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c5736e36-87d2-40fe-982a-005f44f5e444 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:17.240507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c5736e36-87d2-40fe-982a-005f44f5e444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.252779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f01dd1-f7d2-4bc1-8649-6609600970d9 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.340239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 942/1878] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:17.342000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a3588-546d-43ea-b3aa-5bd369926339 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.342283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a3588-546d-43ea-b3aa-5bd369926339 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.342604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a3588-546d-43ea-b3aa-5bd369926339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.352020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.353199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.353505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.353756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] PATH_INFO: `/v3/users/de4526e7c3ea494a89f05ebf5f41da78` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.355969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4526e7c3ea494a89f05ebf5f41da78)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.358943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.370410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-6e0a3588-546d-43ea-b3aa-5bd369926339 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:17.372149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6e0a3588-546d-43ea-b3aa-5bd369926339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.390188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 943/1879] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:17.404985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a3588-546d-43ea-b3aa-5bd369926339 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.415204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a3588-546d-43ea-b3aa-5bd369926339 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.424507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.425296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.426772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.427143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.427928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:17.428334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.430133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.442376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4526e7c3ea494a89f05ebf5f41da78'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.443135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 944/1880] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:17.444366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4526e7c3ea494a89f05ebf5f41da78'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.447543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.454985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4526e7c3ea494a89f05ebf5f41da78'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.456403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4526e7c3ea494a89f05ebf5f41da78'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.459436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a76b5a-6b5a-4f60-9674-59041abbb69a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.470354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.471591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.471591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.473013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.474460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.478908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.483559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cd565b2-a423-4d4d-88bc-f19f66876012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4526e7c3ea494a89f05ebf5f41da78'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.508304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 937/1881] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:17 2022] DELETE /identity/v3/users/de4526e7c3ea494a89f05ebf5f41da78 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:17.508304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8f9d5-5a48-4e17-a01d-1373c30603c3 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.508304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8f9d5-5a48-4e17-a01d-1373c30603c3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.508304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8f9d5-5a48-4e17-a01d-1373c30603c3 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.514323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 938/1882] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:52:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:52:17.515392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e4c8-438b-4766-9af1-78438c99bea3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.515772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e4c8-438b-4766-9af1-78438c99bea3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.516157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e4c8-438b-4766-9af1-78438c99bea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.525507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 945/1883] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:17 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:17.527113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.529114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.540761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a0c5e4c8-438b-4766-9af1-78438c99bea3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:17.542449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a0c5e4c8-438b-4766-9af1-78438c99bea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.615599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:17.626675 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 939/1884] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:17.628115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.628401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.628673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.629510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446254fb-b003-4695-a7ce-5622279e79ad admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.640885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.641809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.642094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.642370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.643021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.645057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.649497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:17.650870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.653746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:17.677133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 946/1885] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:17 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:17.677133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.680209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.696126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0d3de-0153-46e5-9cfd-1f33ae1017aa admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.703498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.704807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.704807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.705428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] PATH_INFO: `/v3/projects/0092148a1a6148e9ad6beb868ad272df` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.706634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0092148a1a6148e9ad6beb868ad272df)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.708590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.727650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 940/1886] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:52:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:17.728883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.730370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.767387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0092148a1a6148e9ad6beb868ad272df', 'request_id': 'req-fe349e8a-ebac-4e61-89f7-5cd759d837c9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.776849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0092148a1a6148e9ad6beb868ad272df', 'request_id': 'req-fe349e8a-ebac-4e61-89f7-5cd759d837c9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.785310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0092148a1a6148e9ad6beb868ad272df', 'request_id': 'req-fe349e8a-ebac-4e61-89f7-5cd759d837c9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.804878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] The token cache is being invalidate because project 0092148a1a6148e9ad6beb868ad272df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:17.805162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.805807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:17.808041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 947/1887] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:17 2022] DELETE /identity/v3/projects/0092148a1a6148e9ad6beb868ad272df => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:17.808667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.810271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.814228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe20c64-9f1b-4a36-a126-b83bb9916266 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.868412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8b3c7-f924-4077-9555-6d40b446a9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.869041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b3c7-f924-4077-9555-6d40b446a9bf service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.869324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b3c7-f924-4077-9555-6d40b446a9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.869752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b3c7-f924-4077-9555-6d40b446a9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.871530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-12f8b3c7-f924-4077-9555-6d40b446a9bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.886764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349e8a-ebac-4e61-89f7-5cd759d837c9 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:17.891697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:17.892348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.892628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.892900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:17.893555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.895325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.912291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 948/1888] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:17.912291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:17.912291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:17.973072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8b3c7-f924-4077-9555-6d40b446a9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:17.977107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8b3c7-f924-4077-9555-6d40b446a9bf service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:17.981973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:17.991346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 941/1889] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:17.993216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e302f-087c-4442-8836-2a25f06b5cdb None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:17.993668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e302f-087c-4442-8836-2a25f06b5cdb None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:17.994007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e302f-087c-4442-8836-2a25f06b5cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.015094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-eb9e302f-087c-4442-8836-2a25f06b5cdb None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:18.016531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-eb9e302f-087c-4442-8836-2a25f06b5cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.025779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04116a-292b-4db0-b10b-a0229fee0a26 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.037465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 942/1890] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:18.037465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e302f-087c-4442-8836-2a25f06b5cdb None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.040726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.041361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.041642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.041909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.044391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e302f-087c-4442-8836-2a25f06b5cdb None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.045248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.049666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b0901-eb51-43c9-9317-73eb270f6e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.050332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0901-eb51-43c9-9317-73eb270f6e36 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.050647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0901-eb51-43c9-9317-73eb270f6e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.050875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0901-eb51-43c9-9317-73eb270f6e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.051973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b0901-eb51-43c9-9317-73eb270f6e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.055007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:18.057667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b0901-eb51-43c9-9317-73eb270f6e36 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.061226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.063833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.076194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 943/1891] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:18.076194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef59ce-94a7-4dce-b687-b2f42b94c5a6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.076194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef59ce-94a7-4dce-b687-b2f42b94c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.076194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef59ce-94a7-4dce-b687-b2f42b94c5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.089333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 949/1892] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:18.090302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.105800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5eef59ce-94a7-4dce-b687-b2f42b94c5a6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:18.107257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5eef59ce-94a7-4dce-b687-b2f42b94c5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.116292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcede55e-8aa8-4933-acfc-1edd95d0e284 service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.124958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 944/1893] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:18.126977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1367b-9a44-4969-a56e-6d7e25acf5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.127284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef59ce-94a7-4dce-b687-b2f42b94c5a6 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.128496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1367b-9a44-4969-a56e-6d7e25acf5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.131325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1367b-9a44-4969-a56e-6d7e25acf5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.131679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1367b-9a44-4969-a56e-6d7e25acf5fd admin admin] PATH_INFO: `/v3/projects/1449f02e43e34e3384690eb632c5b947/users/fe8ce737c21c4c088e58f9dae32d77bf/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.139682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef59ce-94a7-4dce-b687-b2f42b94c5a6 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.144387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.144999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.145425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.145695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.146347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:18.146646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.148532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.160460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1367b-9a44-4969-a56e-6d7e25acf5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1449f02e43e34e3384690eb632c5b947, user_id=fe8ce737c21c4c088e58f9dae32d77bf, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.162423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 945/1894] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:18.165265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1367b-9a44-4969-a56e-6d7e25acf5fd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.166821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.179534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cad16e-9871-4db8-881f-40658af64217 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.187863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 950/1895] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:18 2022] PUT /identity/v3/projects/1449f02e43e34e3384690eb632c5b947/users/fe8ce737c21c4c088e58f9dae32d77bf/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:18.198376 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6848f-7bfc-4e1b-9d37-1eb4200bb8ce None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.198724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6848f-7bfc-4e1b-9d37-1eb4200bb8ce None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.198997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6848f-7bfc-4e1b-9d37-1eb4200bb8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.253676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-25f6848f-7bfc-4e1b-9d37-1eb4200bb8ce None None] MFA Rules not processed for user `fe8ce737c21c4c088e58f9dae32d77bf`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:18.255131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-25f6848f-7bfc-4e1b-9d37-1eb4200bb8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.276636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.277383 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.277749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.278126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.279637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.281409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.315250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 946/1896] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/projects => generated 407 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:18.316547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.346125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 951/1897] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:18.351173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6848f-7bfc-4e1b-9d37-1eb4200bb8ce None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.363072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6848f-7bfc-4e1b-9d37-1eb4200bb8ce None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.403935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bc968-f055-4897-8152-f9843af786e2 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.410319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.411031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.411388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.411719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.412404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.415501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.426991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.428165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.428423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.429030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.429766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.431681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.444345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 947/1898] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/users => generated 304 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:18.445305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.453369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.467737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 952/1899] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:18.467737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.468268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da421afa-56a4-48a7-b442-ab9cfa7a8770 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.501117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:18.513659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:18.528103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.528745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.529019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.529285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] PATH_INFO: `/v3/projects/9a82c282db504701a7ddd68a8bdd0fa3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.530411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a82c282db504701a7ddd68a8bdd0fa3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.531226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffcb4-eba9-4897-8de3-8340aae9f7bf admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.532346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.535849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.536487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.536793 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.537066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.537714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.539359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.549519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 953/1900] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:18.551634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.561581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0588fc9-6c4f-41a5-b531-087e63973a6e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.566830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.567514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.567774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.568076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.568739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.570744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.581188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a82c282db504701a7ddd68a8bdd0fa3', 'request_id': 'req-80917683-fde5-4dee-b85a-1c7278395b2c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:18.587620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a82c282db504701a7ddd68a8bdd0fa3', 'request_id': 'req-80917683-fde5-4dee-b85a-1c7278395b2c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:18.593539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a82c282db504701a7ddd68a8bdd0fa3', 'request_id': 'req-80917683-fde5-4dee-b85a-1c7278395b2c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:18.597769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 954/1901] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/users => generated 298 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:18.602216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.608694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] The token cache is being invalidate because project 9a82c282db504701a7ddd68a8bdd0fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:18.608973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:18.609631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:18.611159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 948/1902] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:18 2022] DELETE /identity/v3/projects/9a82c282db504701a7ddd68a8bdd0fa3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:18.612373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.613830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.614036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9738c2-ba64-48d5-9d41-8284c3dd0bae admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.672161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ef7f0-2961-492f-873e-4c09e90f3cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.672776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ef7f0-2961-492f-873e-4c09e90f3cce admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.673056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ef7f0-2961-492f-873e-4c09e90f3cce admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.673325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ef7f0-2961-492f-873e-4c09e90f3cce admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.673969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ef7f0-2961-492f-873e-4c09e90f3cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.675562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ef7f0-2961-492f-873e-4c09e90f3cce admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.681122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80917683-fde5-4dee-b85a-1c7278395b2c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.685334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 955/1903] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:18.685852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.686464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.686550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69db80-910a-4800-87ab-d8b2b9c5e195 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.686744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.686840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69db80-910a-4800-87ab-d8b2b9c5e195 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.687036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69db80-910a-4800-87ab-d8b2b9c5e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.687114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.687776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.689425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.699876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 949/1904] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:18.700520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.702310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.709857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ad69db80-910a-4800-87ab-d8b2b9c5e195 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:18.711081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ad69db80-910a-4800-87ab-d8b2b9c5e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.714997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16f457-c97b-49d9-869e-1919d751d4e7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.719567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42270c-11df-4b7c-a34d-fde18c98fece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.719847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 956/1905] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:18.720245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42270c-11df-4b7c-a34d-fde18c98fece admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.720526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42270c-11df-4b7c-a34d-fde18c98fece admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.720797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42270c-11df-4b7c-a34d-fde18c98fece admin admin] PATH_INFO: `/v3/projects/26f93ebc5916458b80e78bf940dd91d0/users/c93a613914dc47e3864123459355d139/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.720880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69db80-910a-4800-87ab-d8b2b9c5e195 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.735389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42270c-11df-4b7c-a34d-fde18c98fece admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f93ebc5916458b80e78bf940dd91d0, user_id=c93a613914dc47e3864123459355d139, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.735740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69db80-910a-4800-87ab-d8b2b9c5e195 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.738421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42270c-11df-4b7c-a34d-fde18c98fece admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.740635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b41819-95a6-451f-8847-15f4dcf535ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.741286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41819-95a6-451f-8847-15f4dcf535ff admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.741566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41819-95a6-451f-8847-15f4dcf535ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.741899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41819-95a6-451f-8847-15f4dcf535ff admin admin] PATH_INFO: `/v3/projects/7c62b24e35c4423d9c2068290b6a5a2a/users/d6cdab42834c438eb0b40a2193d75035/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.752259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 950/1906] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:18 2022] PUT /identity/v3/projects/26f93ebc5916458b80e78bf940dd91d0/users/c93a613914dc47e3864123459355d139/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:18.753472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638813e-3549-44c9-ab72-d007a0f3d2db None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.753752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638813e-3549-44c9-ab72-d007a0f3d2db None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.754018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638813e-3549-44c9-ab72-d007a0f3d2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.766804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b41819-95a6-451f-8847-15f4dcf535ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c62b24e35c4423d9c2068290b6a5a2a, user_id=d6cdab42834c438eb0b40a2193d75035, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.769487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b41819-95a6-451f-8847-15f4dcf535ff admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.774208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a638813e-3549-44c9-ab72-d007a0f3d2db None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:18.775710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a638813e-3549-44c9-ab72-d007a0f3d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.784385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 957/1907] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:18 2022] PUT /identity/v3/projects/7c62b24e35c4423d9c2068290b6a5a2a/users/d6cdab42834c438eb0b40a2193d75035/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:18.785581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb4c18-8230-4f20-aaf4-432db4a41d13 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.785853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb4c18-8230-4f20-aaf4-432db4a41d13 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.786128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb4c18-8230-4f20-aaf4-432db4a41d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.809519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:18.813218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9fcb4c18-8230-4f20-aaf4-432db4a41d13 None None] MFA Rules not processed for user `c93a613914dc47e3864123459355d139`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:18.814657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9fcb4c18-8230-4f20-aaf4-432db4a41d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:18.839790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 951/1908] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:18.840745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638813e-3549-44c9-ab72-d007a0f3d2db None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.849890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638813e-3549-44c9-ab72-d007a0f3d2db None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.854548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.855164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.855496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.855767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.856449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.858028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.871982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 952/1909] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:18.871982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.880942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff92ef-4819-4af4-9266-a0a51e1ccc28 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.887287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 958/1910] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:18.887287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb4c18-8230-4f20-aaf4-432db4a41d13 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.887554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.887923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.888370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.888696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.889377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:18.889680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.891346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.896600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb4c18-8230-4f20-aaf4-432db4a41d13 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.901405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.902174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.902453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.902722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] PATH_INFO: `/v3/projects/7c62b24e35c4423d9c2068290b6a5a2a/users/d6cdab42834c438eb0b40a2193d75035/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.904652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c62b24e35c4423d9c2068290b6a5a2a, user_id=d6cdab42834c438eb0b40a2193d75035, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.906694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 953/1911] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:18.907547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.910014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:18.918058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1262596-d874-4473-8233-2725d6ba7ef8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:18.923574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 959/1912] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:18 2022] PUT /identity/v3/projects/7c62b24e35c4423d9c2068290b6a5a2a/users/d6cdab42834c438eb0b40a2193d75035/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:18.924791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:18.930456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:18.987786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98fe26-7a05-429b-9668-3a8b5f4d2e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:18.988460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fe26-7a05-429b-9668-3a8b5f4d2e88 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:18.988761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fe26-7a05-429b-9668-3a8b5f4d2e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:18.989430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98fe26-7a05-429b-9668-3a8b5f4d2e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:18.989657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98fe26-7a05-429b-9668-3a8b5f4d2e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:18.991770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98fe26-7a05-429b-9668-3a8b5f4d2e88 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.005569 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14619a9c-522f-4747-b8b1-b78500861a65 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.017153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.018070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.018504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.018780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.019422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.020681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 954/1913] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/projects => generated 409 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:19.021374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.021952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6d0f1-bdcf-4d55-a5ec-89b090f38307 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.022230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6d0f1-bdcf-4d55-a5ec-89b090f38307 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.022528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6d0f1-bdcf-4d55-a5ec-89b090f38307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.051077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-87f6d0f1-bdcf-4d55-a5ec-89b090f38307 None None] MFA Rules not processed for user `d6cdab42834c438eb0b40a2193d75035`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:19.052406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-87f6d0f1-bdcf-4d55-a5ec-89b090f38307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:19.054098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 960/1914] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:18 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:19.055027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.120898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cba79-8a37-4968-b085-27574b862cc2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.126309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.126470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.126674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.126947 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.127592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.129537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.133392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 955/1915] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:19.138756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6d0f1-bdcf-4d55-a5ec-89b090f38307 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.150507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6d0f1-bdcf-4d55-a5ec-89b090f38307 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.155097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 961/1916] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:19 2022] POST /identity/v3/users => generated 311 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:19.155959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.158854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.159691 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.159868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:19.159997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.160287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.160914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.162950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.190378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 956/1917] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:19 2022] POST /identity/v3/users => generated 288 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:19.190378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.201552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05187b1b-6105-4518-9c2b-6f13b5e2666b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.207407 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.208176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.208704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.208913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.209632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.211348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.212914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:19.219818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef8b22-69a1-43a0-8627-67f45f06a2b1 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.223812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.224476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.224772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.225143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 957/1918] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:19.225969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.228317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.236657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6909-bb17-4eb6-b702-cde02a6448ca admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.241776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.242364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.242585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.243007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.243396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.244058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.244253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.245964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.253963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 962/1919] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:19.254751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.258100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 958/1920] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:19.263310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.269683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e5bf5-762f-42ac-9674-03121a3e48ca service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.274727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eee4c6-3eab-4d8c-af50-d0c06d4593d7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.275195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.276150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.276566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.277015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] PATH_INFO: `/v3/projects/c1947176708b4a32b0953565e1fdad45/users/b096bb866d3049bab13f4b24b5a304ab/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.280137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.280858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.281213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.281539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.282266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.284034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.295627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1947176708b4a32b0953565e1fdad45, user_id=b096bb866d3049bab13f4b24b5a304ab, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.298706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.307991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 959/1921] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:19.312816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.316656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:19.325995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 963/1922] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:19 2022] PUT /identity/v3/projects/c1947176708b4a32b0953565e1fdad45/users/b096bb866d3049bab13f4b24b5a304ab/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:19.330169 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.337121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe6b2a-347d-48c9-afda-219ade04f269 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.417630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e35cb-02df-48fb-8d4d-d8abfdd43223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.418722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e35cb-02df-48fb-8d4d-d8abfdd43223 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.419154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e35cb-02df-48fb-8d4d-d8abfdd43223 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.419571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e35cb-02df-48fb-8d4d-d8abfdd43223 admin admin] PATH_INFO: `/v3/projects/6aae9acee72c40df93a948a7dfe006ae/users/d81f4a50a67d4dd7bf8e590c924db1ef/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.426996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513f0b3-7a2e-455a-a866-6beaed4a8587 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.434931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.436402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.436950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.437977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.438920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.441427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.443998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e35cb-02df-48fb-8d4d-d8abfdd43223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aae9acee72c40df93a948a7dfe006ae, user_id=d81f4a50a67d4dd7bf8e590c924db1ef, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.446981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e35cb-02df-48fb-8d4d-d8abfdd43223 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.464082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 964/1923] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:19.465197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.471518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 960/1924] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:19 2022] PUT /identity/v3/projects/6aae9acee72c40df93a948a7dfe006ae/users/d81f4a50a67d4dd7bf8e590c924db1ef/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:19.485380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df742ca0-4799-4707-898b-c9fb1536e9d8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.485745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df742ca0-4799-4707-898b-c9fb1536e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.486112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df742ca0-4799-4707-898b-c9fb1536e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.488180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7fb3d-1fb8-4ec7-bc02-77d6e406cb71 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.516658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-df742ca0-4799-4707-898b-c9fb1536e9d8 None None] MFA Rules not processed for user `d81f4a50a67d4dd7bf8e590c924db1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:19.518023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-df742ca0-4799-4707-898b-c9fb1536e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:19.569090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.569765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.570071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.570345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] PATH_INFO: `/v3/projects/c1947176708b4a32b0953565e1fdad45/users/b096bb866d3049bab13f4b24b5a304ab/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.572265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1947176708b4a32b0953565e1fdad45, user_id=b096bb866d3049bab13f4b24b5a304ab, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.574892 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.587966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 965/1925] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:19 2022] PUT /identity/v3/projects/c1947176708b4a32b0953565e1fdad45/users/b096bb866d3049bab13f4b24b5a304ab/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:19.588828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.601352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 961/1926] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:19.602627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bf52f-f372-4f2a-bf6d-044c1770b10d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.602909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bf52f-f372-4f2a-bf6d-044c1770b10d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.603183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bf52f-f372-4f2a-bf6d-044c1770b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.631586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6a7bf52f-f372-4f2a-bf6d-044c1770b10d None None] MFA Rules not processed for user `b096bb866d3049bab13f4b24b5a304ab`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:19.632925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6a7bf52f-f372-4f2a-bf6d-044c1770b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:19.651081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63b646-afdd-436f-a1c5-3912ee78ef14 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.654822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.655443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.655719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.656027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.657811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:19.667381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:19.701229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:19.703067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 962/1927] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:19.703718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bf52f-f372-4f2a-bf6d-044c1770b10d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.721262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.722722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.733451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 966/1928] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:19.734483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.736130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:19.747231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8817797-4d7b-4c4f-93f4-7a05e2c61c7b service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.752136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.752717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.753009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.753257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.764585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.766076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.777412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bf52f-f372-4f2a-bf6d-044c1770b10d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.777736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 967/1929] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:19.782113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.782846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.783009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.783280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.783919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.786096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.817377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 963/1930] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:19 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:19.821624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.832731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69e7c-be83-4d53-82b2-34dd8f29eaab service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.905069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.905683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.905960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.906227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.906861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.908803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.932548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 968/1931] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:19 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:19.937018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.948845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c507-5106-464f-abe9-c0961dc26212 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.953895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.954642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.954922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.955190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:19.955838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:19.957538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:19.968693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 964/1932] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:19.972942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:19.983080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae0a11-8b9d-4fb8-a169-46b94134dc74 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:19.991227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:19.991966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:19.992242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:19.992512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] PATH_INFO: `/v3/projects/47db851d76ca40f79a30bbc5d5d44e0f/users/de41cfe4c4fd4f488c178ae4131a62b1/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.000887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:20.007960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47db851d76ca40f79a30bbc5d5d44e0f, user_id=de41cfe4c4fd4f488c178ae4131a62b1, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:20.010619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:20.027129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 969/1933] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:19 2022] PUT /identity/v3/projects/47db851d76ca40f79a30bbc5d5d44e0f/users/de41cfe4c4fd4f488c178ae4131a62b1/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:20.031210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:20.063163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31992-e336-4ea0-af80-9321a0ff9edc admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:20.095183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdd9ac-3547-4894-ba68-abeb1d09cf17 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:20.100571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:20.101522 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.101900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.102293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.103239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:20.105146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:20.120732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 970/1934] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:20.120732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:20.122265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:20.122896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.123170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.123444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.133093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe04223-7110-4faf-b390-948799bd562b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:20.138038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:20.138547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:20.138699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.138951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.139221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] PATH_INFO: `/v3/users/fe8ce737c21c4c088e58f9dae32d77bf` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.140385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8ce737c21c4c088e58f9dae32d77bf)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:20.142296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:20.178181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:20.188954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:20.192668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:20.202468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 965/1935] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:19 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:20.202995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:20.205912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8ce737c21c4c088e58f9dae32d77bf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:20.206589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8ce737c21c4c088e58f9dae32d77bf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:20.207384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8ce737c21c4c088e58f9dae32d77bf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:20.213565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2170d14-7d7a-4fd4-8dd2-bf2f7a39dac9 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:20.216156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8ce737c21c4c088e58f9dae32d77bf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:20.218290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154a225-351b-49a2-b58f-9b7d29b9811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:20.219008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154a225-351b-49a2-b58f-9b7d29b9811d admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.219350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154a225-351b-49a2-b58f-9b7d29b9811d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.219684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154a225-351b-49a2-b58f-9b7d29b9811d admin admin] PATH_INFO: `/v3/projects/47db851d76ca40f79a30bbc5d5d44e0f/users/de41cfe4c4fd4f488c178ae4131a62b1/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.221926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154a225-351b-49a2-b58f-9b7d29b9811d admin admin] RBAC: Authorizing `identity:create_grant(project_id=47db851d76ca40f79a30bbc5d5d44e0f, user_id=de41cfe4c4fd4f488c178ae4131a62b1, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:20.224596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154a225-351b-49a2-b58f-9b7d29b9811d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:20.234071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d39fd43b-8da8-4d1a-9851-c01b9b32e938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8ce737c21c4c088e58f9dae32d77bf'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:20.235540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 966/1936] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:20 2022] PUT /identity/v3/projects/47db851d76ca40f79a30bbc5d5d44e0f/users/de41cfe4c4fd4f488c178ae4131a62b1/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:20.246919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.247279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.247636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.262988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 971/1937] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:20 2022] DELETE /identity/v3/users/fe8ce737c21c4c088e58f9dae32d77bf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:20.266034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.266383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.267046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.282961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] MFA Rules not processed for user `de41cfe4c4fd4f488c178ae4131a62b1`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:20.284218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:20.303025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:20.304488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:20.385390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 972/1938] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:20.392115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 967/1939] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:20.397232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:20.399847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:20.426045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:20.427590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:20.502934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10edc69-897e-433e-a974-a0118b0f11e7 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:20.512548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:20.512682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c66b6-1e31-46c9-a6ae-7546275ce702 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:20.515125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.515453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.515873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.517122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:20.517965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.518412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.518857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.520970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:20.528887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:20.530349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:20.545547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 973/1940] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:20.649840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:20.694030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:20.696897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:20.725798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 968/1941] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:52:20.974819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:20.976362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:20.989775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105b282-ebde-4a97-8895-698f42c648d8 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:20.995147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:20.996007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:20.996474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:20.996882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] PATH_INFO: `/v3/projects/1449f02e43e34e3384690eb632c5b947` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:20.998380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1449f02e43e34e3384690eb632c5b947)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:21.000358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:21.049024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:21.057342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1449f02e43e34e3384690eb632c5b947', 'request_id': 'req-282fecb5-8886-45a0-b1e2-8bc31655b207'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.064688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1449f02e43e34e3384690eb632c5b947', 'request_id': 'req-282fecb5-8886-45a0-b1e2-8bc31655b207'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.070940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1449f02e43e34e3384690eb632c5b947', 'request_id': 'req-282fecb5-8886-45a0-b1e2-8bc31655b207'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.085199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] The token cache is being invalidate because project 1449f02e43e34e3384690eb632c5b947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:21.085680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.086356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.087938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 974/1942] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:20 2022] DELETE /identity/v3/projects/1449f02e43e34e3384690eb632c5b947 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:21.091900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:21.105009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19755a-972e-4c08-99c4-23ad5f971792 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:21.173568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:21.174299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:21.174581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:21.174852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] PATH_INFO: `/v3/users/d6cdab42834c438eb0b40a2193d75035` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:21.176017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6cdab42834c438eb0b40a2193d75035)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:21.177956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:21.226374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6cdab42834c438eb0b40a2193d75035'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.226935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6cdab42834c438eb0b40a2193d75035'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.233664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6cdab42834c438eb0b40a2193d75035'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.234277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6cdab42834c438eb0b40a2193d75035'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.246833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0fdccf2f-cf9d-42c1-9df3-c8cb847c36b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6cdab42834c438eb0b40a2193d75035'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.260871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 969/1943] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:21 2022] DELETE /identity/v3/users/d6cdab42834c438eb0b40a2193d75035 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:21.590609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:21.593719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:21.619784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fecb5-8886-45a0-b1e2-8bc31655b207 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:21.692229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:21.692898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:21.693179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:21.693449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] PATH_INFO: `/v3/projects/7c62b24e35c4423d9c2068290b6a5a2a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:21.694670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c62b24e35c4423d9c2068290b6a5a2a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:21.696853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:21.756722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c62b24e35c4423d9c2068290b6a5a2a', 'request_id': 'req-9a72a5a3-dae1-4668-8f5c-b012bc94635c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.763435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c62b24e35c4423d9c2068290b6a5a2a', 'request_id': 'req-9a72a5a3-dae1-4668-8f5c-b012bc94635c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.770273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c62b24e35c4423d9c2068290b6a5a2a', 'request_id': 'req-9a72a5a3-dae1-4668-8f5c-b012bc94635c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.784598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] The token cache is being invalidate because project 7c62b24e35c4423d9c2068290b6a5a2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:21.784891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.785535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9a72a5a3-dae1-4668-8f5c-b012bc94635c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:21.787279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 975/1944] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:21 2022] DELETE /identity/v3/projects/7c62b24e35c4423d9c2068290b6a5a2a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:21.792129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedb0fa-2abf-43d1-96bb-310782ca60c9 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:21.792650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedb0fa-2abf-43d1-96bb-310782ca60c9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:21.792650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedb0fa-2abf-43d1-96bb-310782ca60c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:21.818397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5dedb0fa-2abf-43d1-96bb-310782ca60c9 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:21.819606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5dedb0fa-2abf-43d1-96bb-310782ca60c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:21.894515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 970/1945] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:21.903343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453c16a-54b3-43b7-9f65-7954acf425d4 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:21.903636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453c16a-54b3-43b7-9f65-7954acf425d4 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:21.904038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453c16a-54b3-43b7-9f65-7954acf425d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:21.924306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9453c16a-54b3-43b7-9f65-7954acf425d4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:21.925542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9453c16a-54b3-43b7-9f65-7954acf425d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:21.938121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 976/1946] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:21.942025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedb0fa-2abf-43d1-96bb-310782ca60c9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:21.954129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:21.957335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedb0fa-2abf-43d1-96bb-310782ca60c9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:21.980089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:21.980681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:21.981740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:21.981905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:21.982530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:21.982881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:21.985504 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.004149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 971/1947] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:22.006086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453c16a-54b3-43b7-9f65-7954acf425d4 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.016296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453c16a-54b3-43b7-9f65-7954acf425d4 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.021246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.021846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.022126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.022400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.023210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.025178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.053280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 977/1948] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:22 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:22.057801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.068248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200983c-eee0-497e-94d2-5706d9f09741 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.125936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.126548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.126820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.127088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.127726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.130150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.164500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 972/1949] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:22 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:22.168931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.179863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66e16-d9fd-4a7d-beac-dbc7988b7c9a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.185683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.186281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.186580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.186849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.187501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.189111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.202156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 978/1950] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:22.207728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.218766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604c454-31fc-4a28-8224-1d89face6ae0 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.223354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831199fe-ec94-415b-8e86-4b7fe08598c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.223994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831199fe-ec94-415b-8e86-4b7fe08598c7 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.224268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831199fe-ec94-415b-8e86-4b7fe08598c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.224535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831199fe-ec94-415b-8e86-4b7fe08598c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.225174 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831199fe-ec94-415b-8e86-4b7fe08598c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.226824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831199fe-ec94-415b-8e86-4b7fe08598c7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.237421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 973/1951] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:22.241594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.251479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cafb7-5329-4fbf-a585-6a25e021bb8f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.256285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.256868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.257140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.257471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] PATH_INFO: `/v3/projects/91897b838c754c04bfcb3e550dc8e5e8/users/f3810c220dcd46f89caf5a565c44d07a/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.273266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91897b838c754c04bfcb3e550dc8e5e8, user_id=f3810c220dcd46f89caf5a565c44d07a, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.275911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.279264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:22.292945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 979/1952] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:22 2022] PUT /identity/v3/projects/91897b838c754c04bfcb3e550dc8e5e8/users/f3810c220dcd46f89caf5a565c44d07a/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:22.303350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33efd4-279b-4744-974e-30d5e109f9ec None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.303634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33efd4-279b-4744-974e-30d5e109f9ec None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.303906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33efd4-279b-4744-974e-30d5e109f9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.332410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-1d33efd4-279b-4744-974e-30d5e109f9ec None None] MFA Rules not processed for user `f3810c220dcd46f89caf5a565c44d07a`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:22.333603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1d33efd4-279b-4744-974e-30d5e109f9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:22.399302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 974/1953] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:22.410891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.412970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:22.471533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d37425-4fdb-4070-aaec-90cd6289f311 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.475448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffcf00-7b7d-42ac-bf53-5e5d22d8581f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.476113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffcf00-7b7d-42ac-bf53-5e5d22d8581f service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.476385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffcf00-7b7d-42ac-bf53-5e5d22d8581f service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.476651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffcf00-7b7d-42ac-bf53-5e5d22d8581f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.485126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffcf00-7b7d-42ac-bf53-5e5d22d8581f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.486570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffcf00-7b7d-42ac-bf53-5e5d22d8581f service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.496166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 980/1954] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:22.515731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33efd4-279b-4744-974e-30d5e109f9ec None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.594857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:22.599168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33efd4-279b-4744-974e-30d5e109f9ec None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.604719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.605365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.605639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.605903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] PATH_INFO: `/v3/users/f3810c220dcd46f89caf5a565c44d07a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.607034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3810c220dcd46f89caf5a565c44d07a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.609147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.658961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3810c220dcd46f89caf5a565c44d07a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:22.659623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3810c220dcd46f89caf5a565c44d07a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:22.667318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3810c220dcd46f89caf5a565c44d07a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:22.668030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3810c220dcd46f89caf5a565c44d07a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:22.680088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3810c220dcd46f89caf5a565c44d07a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:22.694404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 975/1955] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:22 2022] DELETE /identity/v3/users/f3810c220dcd46f89caf5a565c44d07a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:22.699058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.699498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.699635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.718628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:22.719819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:22.781327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 981/1956] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:22.791427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:22.792968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:22.865822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716f8d3-e967-4164-96ea-8589ca640848 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:22.869408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed07bee-a001-4166-9242-de40297712d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:22.870023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed07bee-a001-4166-9242-de40297712d6 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:22.870301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed07bee-a001-4166-9242-de40297712d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:22.870567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed07bee-a001-4166-9242-de40297712d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:22.879875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed07bee-a001-4166-9242-de40297712d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:22.881360 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed07bee-a001-4166-9242-de40297712d6 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:22.890771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 976/1957] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:23.238103 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:23.239642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:23.252820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72621c2-17c5-47d1-a221-d56fcc56eb97 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:23.268293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:23.268293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.268293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.268293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] PATH_INFO: `/v3/projects/91897b838c754c04bfcb3e550dc8e5e8` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.268948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] RBAC: Authorizing `identity:delete_project(project_id=91897b838c754c04bfcb3e550dc8e5e8)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:23.268948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:23.319505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91897b838c754c04bfcb3e550dc8e5e8', 'request_id': 'req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:23.326448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91897b838c754c04bfcb3e550dc8e5e8', 'request_id': 'req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:23.332702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91897b838c754c04bfcb3e550dc8e5e8', 'request_id': 'req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:23.347532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] The token cache is being invalidate because project 91897b838c754c04bfcb3e550dc8e5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:23.347819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:23.348460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b03c745e-9f2f-48d5-b7dc-dd9c61f2413a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:23.349976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 982/1958] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:23 2022] DELETE /identity/v3/projects/91897b838c754c04bfcb3e550dc8e5e8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:23.355002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b05-b174-4676-a2ae-8784544a268a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.355312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b05-b174-4676-a2ae-8784544a268a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.355584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4b05-b174-4676-a2ae-8784544a268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.376686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0b2e4b05-b174-4676-a2ae-8784544a268a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:23.377902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0b2e4b05-b174-4676-a2ae-8784544a268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:23.443722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 977/1959] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:23.451706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37877f4-c93b-47c4-a5d1-125fbf9938cf None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.452019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37877f4-c93b-47c4-a5d1-125fbf9938cf None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.452312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37877f4-c93b-47c4-a5d1-125fbf9938cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.473494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f37877f4-c93b-47c4-a5d1-125fbf9938cf None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:23.474703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f37877f4-c93b-47c4-a5d1-125fbf9938cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:23.483001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 983/1960] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:23.487766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4b05-b174-4676-a2ae-8784544a268a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:23.498087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4b05-b174-4676-a2ae-8784544a268a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:23.503215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:23.503803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.504112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.504415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.505060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:23.505372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:23.506967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:23.519801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 978/1961] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:23.526235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37877f4-c93b-47c4-a5d1-125fbf9938cf None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:23.541623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37877f4-c93b-47c4-a5d1-125fbf9938cf None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:23.547925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:23.548577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.548848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.549112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.549745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:23.551645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:23.568567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:23.588831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 984/1962] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:23 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:23.590017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:23.600294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6874d-101a-400a-9389-c8d11fd2f24b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:23.607969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:23.663713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:23.664377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.664656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.664927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.665559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:23.667481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:23.689738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 979/1963] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:23 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:23.694133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:23.704989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4797c-30d2-4bf6-9c88-392bb1e16cc8 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:23.710422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:23.711030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.711353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.711630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.712317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:23.713887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:23.731683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 985/1964] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:23.733021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:23.743668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11605b-27ee-4e8f-a78e-3c1f4e36dd5f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:23.750746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9f45a-a622-4a3c-afd6-07d7b2dde220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:23.751371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f45a-a622-4a3c-afd6-07d7b2dde220 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.751652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f45a-a622-4a3c-afd6-07d7b2dde220 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.751948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9f45a-a622-4a3c-afd6-07d7b2dde220 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.752599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9f45a-a622-4a3c-afd6-07d7b2dde220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:23.754210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9f45a-a622-4a3c-afd6-07d7b2dde220 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:23.767677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 980/1965] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:23.772215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:23.783204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dedb0d-813d-4ab7-9e44-5643acf6afe2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:23.788905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:23.789587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.789864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.790134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] PATH_INFO: `/v3/projects/2a22714b2eef478f967f243e351dbb1b/users/cec4a3cc81a449a9adf2d73ea454499d/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.806426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a22714b2eef478f967f243e351dbb1b, user_id=cec4a3cc81a449a9adf2d73ea454499d, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:23.809166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:23.825922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 986/1966] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:23 2022] PUT /identity/v3/projects/2a22714b2eef478f967f243e351dbb1b/users/cec4a3cc81a449a9adf2d73ea454499d/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:23.836539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d247c-02d2-4662-9ff2-8e1c3493e303 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:23.836942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d247c-02d2-4662-9ff2-8e1c3493e303 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:23.837092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d247c-02d2-4662-9ff2-8e1c3493e303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:23.871411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-cc6d247c-02d2-4662-9ff2-8e1c3493e303 None None] MFA Rules not processed for user `cec4a3cc81a449a9adf2d73ea454499d`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:23.872679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-cc6d247c-02d2-4662-9ff2-8e1c3493e303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:23.947252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 981/1967] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:23.952083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.020062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7617c8b-b64b-46c3-abde-dc3bf1d2d981 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.024794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.025403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.025678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.025948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.026595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.028528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.069124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 987/1968] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:23 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:24.073385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d247c-02d2-4662-9ff2-8e1c3493e303 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.084345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d247c-02d2-4662-9ff2-8e1c3493e303 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.143119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.143746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.144053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.144327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.144981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.146916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.171676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 982/1969] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:24 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:24.177365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.189412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e2d2e-d725-4005-b4e2-046bfe75ae51 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.196893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.197533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.197802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.198076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.198753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.200389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.211963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 988/1970] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:24.216151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.222702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:24.226804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d430d6-979d-4298-a188-cbdcf95be163 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.233021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8482f-1ab6-45fe-b60a-0f6a850765bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.233653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8482f-1ab6-45fe-b60a-0f6a850765bc admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.233933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8482f-1ab6-45fe-b60a-0f6a850765bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.234204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8482f-1ab6-45fe-b60a-0f6a850765bc admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.234855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8482f-1ab6-45fe-b60a-0f6a850765bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.236555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8482f-1ab6-45fe-b60a-0f6a850765bc admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.247122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 983/1971] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:24.251366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.262252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32abe0a-f11e-4fd2-8ff3-ea8722a435fc admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.268228 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.268924 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.269187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.269544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] PATH_INFO: `/v3/projects/121ab10fdfe54853bcffa73f57fc0b40/users/1097bf0fd7704e7c93fba36aca0b6229/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.305806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121ab10fdfe54853bcffa73f57fc0b40, user_id=1097bf0fd7704e7c93fba36aca0b6229, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.308549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.323454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 989/1972] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:24 2022] PUT /identity/v3/projects/121ab10fdfe54853bcffa73f57fc0b40/users/1097bf0fd7704e7c93fba36aca0b6229/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:24.333478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df06f8c-1d32-44c7-8dee-8f6124270887 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.333770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df06f8c-1d32-44c7-8dee-8f6124270887 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.334047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df06f8c-1d32-44c7-8dee-8f6124270887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.364225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6df06f8c-1d32-44c7-8dee-8f6124270887 None None] MFA Rules not processed for user `1097bf0fd7704e7c93fba36aca0b6229`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:24.365518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6df06f8c-1d32-44c7-8dee-8f6124270887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:24.435444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 984/1973] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:24.439754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.507825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd818ce4-a72b-4545-9c8b-a67ae49879a8 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.513384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.513998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.514276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.514548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.515199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.517212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.534454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:24.553899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 990/1974] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:24 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:24.558187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df06f8c-1d32-44c7-8dee-8f6124270887 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.569600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df06f8c-1d32-44c7-8dee-8f6124270887 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.630922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.631540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.631841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.632139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.632764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.634655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.658705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 985/1975] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:24 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:24.662648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.673625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb560f76-6f00-4a92-8e27-14a9914fd65f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.678927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.679525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.679929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.680198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.680840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.682409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.693881 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 991/1976] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:24.697993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.708756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f01a9c-a64a-4a57-b81a-e9f9d80a9e83 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.713576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.714238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.714528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.714790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] PATH_INFO: `/v3/projects/f64336ca2b374f8a803fbcff46a8d18f/users/455b53300f334ee8aa733d574c241052/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.730049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64336ca2b374f8a803fbcff46a8d18f, user_id=455b53300f334ee8aa733d574c241052, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.732735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.747394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 986/1977] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:24 2022] PUT /identity/v3/projects/f64336ca2b374f8a803fbcff46a8d18f/users/455b53300f334ee8aa733d574c241052/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:24.751514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.761858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f3193-4962-43b7-8891-014e34866faf admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.816949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11faf8-8c54-4463-a9ad-7915a8c40fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.817553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11faf8-8c54-4463-a9ad-7915a8c40fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.817825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11faf8-8c54-4463-a9ad-7915a8c40fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.818094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11faf8-8c54-4463-a9ad-7915a8c40fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.818729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11faf8-8c54-4463-a9ad-7915a8c40fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.820374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11faf8-8c54-4463-a9ad-7915a8c40fc7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.829972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 992/1978] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:24.834190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:24.844622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64939da5-93ed-449b-bcd0-4d043bb4703c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:24.849418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:24.850063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.850340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.850604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] PATH_INFO: `/v3/projects/f64336ca2b374f8a803fbcff46a8d18f/users/455b53300f334ee8aa733d574c241052/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.852551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64336ca2b374f8a803fbcff46a8d18f, user_id=455b53300f334ee8aa733d574c241052, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:24.855172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:24.865865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 987/1979] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:24 2022] PUT /identity/v3/projects/f64336ca2b374f8a803fbcff46a8d18f/users/455b53300f334ee8aa733d574c241052/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:24.876981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:24.877398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:24.877504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:24.910836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] MFA Rules not processed for user `455b53300f334ee8aa733d574c241052`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:24.912096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:24.993032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 993/1980] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:25.003563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:25.005387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:25.047172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:25.066469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb7a8-7990-4979-8510-fd73365263be admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:25.070224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:25.070996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:25.071568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:25.071568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:25.149835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:25.151276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:25.160306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 988/1981] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:34.168777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:34.170337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:34.182487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e67d6-c80c-4cbc-806d-a688047bfe1f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:34.186235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:34.186922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:34.187260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:34.187589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:34.248619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:34.250050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:34.259147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 994/1982] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:34.898815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:34.915099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f5186-307b-4b56-926e-ab77f32eaec6 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:34.921742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:34.922458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:34.922821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:34.923160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:34.935117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:34.936753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:34.948501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 989/1983] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:39.233688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:39.238332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:39.245666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e820-1567-43da-abf9-9ab96928e62e service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:39.250474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84e911-b142-4ae6-9c91-54e89c02b0f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:39.251209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e911-b142-4ae6-9c91-54e89c02b0f5 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:39.251516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e911-b142-4ae6-9c91-54e89c02b0f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:39.251783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e911-b142-4ae6-9c91-54e89c02b0f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:39.253747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-db84e911-b142-4ae6-9c91-54e89c02b0f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:39.378181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e911-b142-4ae6-9c91-54e89c02b0f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:39.379722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e911-b142-4ae6-9c91-54e89c02b0f5 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:39.391986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 995/1984] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:39.591492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:39.593678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:39.693022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06d189-40ae-4e36-aebd-490131ba77b5 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:39.698517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:39.699284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:39.699565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:39.699961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] PATH_INFO: `/v3/users/c93a613914dc47e3864123459355d139` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:39.701533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93a613914dc47e3864123459355d139)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:39.704722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:39.791676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93a613914dc47e3864123459355d139'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:39.794418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93a613914dc47e3864123459355d139'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:39.806733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93a613914dc47e3864123459355d139'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:39.808338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93a613914dc47e3864123459355d139'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:39.851896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93a613914dc47e3864123459355d139'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:39.857072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:39.879779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 990/1985] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:39 2022] DELETE /identity/v3/users/c93a613914dc47e3864123459355d139 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:39.884283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:39.884875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:39.885198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:39.926602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:39.927830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:40.068909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 996/1986] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:40.081776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:40.084096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:40.242359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e0640-f825-47df-8da7-9d9e7f99b469 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:40.250059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:40.250826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:40.251316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:40.251911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:40.282448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:40.284317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:40.300973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 991/1987] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:40.627407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:40.629084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:40.642886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392fc67-498e-498a-bdcb-00b480bac18c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:40.650562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:40.652625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:40.653066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:40.653404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] PATH_INFO: `/v3/projects/26f93ebc5916458b80e78bf940dd91d0` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:40.655528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f93ebc5916458b80e78bf940dd91d0)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:40.657772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:40.711788 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f93ebc5916458b80e78bf940dd91d0', 'request_id': 'req-0761444d-5294-4fac-80b9-e2bef199fe27'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.718208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f93ebc5916458b80e78bf940dd91d0', 'request_id': 'req-0761444d-5294-4fac-80b9-e2bef199fe27'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.723665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f93ebc5916458b80e78bf940dd91d0', 'request_id': 'req-0761444d-5294-4fac-80b9-e2bef199fe27'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.737501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] The token cache is being invalidate because project 26f93ebc5916458b80e78bf940dd91d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:40.737796 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.738410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.740018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 997/1988] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:40 2022] DELETE /identity/v3/projects/26f93ebc5916458b80e78bf940dd91d0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:40.744305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:40.753880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76011c1-852b-4fb2-9982-276789e94f13 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:40.815673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:40.816367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:40.816651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:40.816911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] PATH_INFO: `/v3/users/b096bb866d3049bab13f4b24b5a304ab` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:40.818063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b096bb866d3049bab13f4b24b5a304ab)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:40.820031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:40.880437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b096bb866d3049bab13f4b24b5a304ab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.882085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b096bb866d3049bab13f4b24b5a304ab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.890435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b096bb866d3049bab13f4b24b5a304ab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.891158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b096bb866d3049bab13f4b24b5a304ab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.903802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0c8c871d-5ae3-44fc-8d42-9462ee1d2d7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b096bb866d3049bab13f4b24b5a304ab'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:40.918787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 992/1989] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:40 2022] DELETE /identity/v3/users/b096bb866d3049bab13f4b24b5a304ab => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:41.215470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:41.217007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:41.246735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761444d-5294-4fac-80b9-e2bef199fe27 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:41.286721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:41.373593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:41.374685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.375216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.375746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] PATH_INFO: `/v3/projects/c1947176708b4a32b0953565e1fdad45` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.377764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1947176708b4a32b0953565e1fdad45)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:41.381739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:41.460479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1947176708b4a32b0953565e1fdad45', 'request_id': 'req-b0748ac1-4391-4683-977c-7e9516116aa4'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:41.467674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1947176708b4a32b0953565e1fdad45', 'request_id': 'req-b0748ac1-4391-4683-977c-7e9516116aa4'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:41.478793 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1947176708b4a32b0953565e1fdad45', 'request_id': 'req-b0748ac1-4391-4683-977c-7e9516116aa4'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:41.499596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] The token cache is being invalidate because project c1947176708b4a32b0953565e1fdad45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:41.500714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:41.501592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0748ac1-4391-4683-977c-7e9516116aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:41.504509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 998/1990] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:41 2022] DELETE /identity/v3/projects/c1947176708b4a32b0953565e1fdad45 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:41.509463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286d49-8fda-4262-8840-1a118ba88ba4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.509908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286d49-8fda-4262-8840-1a118ba88ba4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.510277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286d49-8fda-4262-8840-1a118ba88ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.532526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-16286d49-8fda-4262-8840-1a118ba88ba4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:41.534093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-16286d49-8fda-4262-8840-1a118ba88ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:41.554422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:41.627083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fafb04-413e-49b0-888b-6c8dba6c5366 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.627399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fafb04-413e-49b0-888b-6c8dba6c5366 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.627669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fafb04-413e-49b0-888b-6c8dba6c5366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.632927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 993/1991] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:41.652367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-22fafb04-413e-49b0-888b-6c8dba6c5366 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:41.653863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-22fafb04-413e-49b0-888b-6c8dba6c5366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:41.662580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 999/1992] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:41.667278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16286d49-8fda-4262-8840-1a118ba88ba4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:41.680571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16286d49-8fda-4262-8840-1a118ba88ba4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:41.687165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:41.687985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.688264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.688531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.689236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:41.689632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:41.691234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:41.729098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 994/1993] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:41.730327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fafb04-413e-49b0-888b-6c8dba6c5366 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:41.749133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fafb04-413e-49b0-888b-6c8dba6c5366 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:41.756795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:41.757873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.759211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.760551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.761252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:41.763856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:41.808717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1000/1994] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:41 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:41.812906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:41.824442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb773de9-7a94-476d-868e-b41d5d49132f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:41.879810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:41.880454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.880725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.881003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.881660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:41.883598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:41.906302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 995/1995] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:41 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:41.910050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:41.920497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ecc6f-ee8e-4871-a995-f6b9e1f9523f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:41.925304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:41.926321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.926608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.926884 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.927688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:41.929319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:41.939684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1001/1996] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:41.943704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:41.955109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d78e7-73b3-4935-9d88-ceb606f188b7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:41.960051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e031af-1bff-47ef-be65-2f69fe57111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:41.960774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e031af-1bff-47ef-be65-2f69fe57111a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.961058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e031af-1bff-47ef-be65-2f69fe57111a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.961428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e031af-1bff-47ef-be65-2f69fe57111a admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:41.962086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e031af-1bff-47ef-be65-2f69fe57111a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:41.963994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e031af-1bff-47ef-be65-2f69fe57111a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:41.980635 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 996/1997] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:41.982007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:41.992879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edbe2-4e6e-4901-90e6-543eef961648 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:41.997782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:41.998448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:41.998723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:41.998987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] PATH_INFO: `/v3/projects/e011aab360e94dee8dcafc96d73811e7/users/62742a74bc6c47d386cc9a6f460ce832/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.015835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e011aab360e94dee8dcafc96d73811e7, user_id=62742a74bc6c47d386cc9a6f460ce832, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.019632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.037739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1002/1998] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:41 2022] PUT /identity/v3/projects/e011aab360e94dee8dcafc96d73811e7/users/62742a74bc6c47d386cc9a6f460ce832/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:42.049712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ff76-7321-44c8-acac-cf3af5b59c8a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.050002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ff76-7321-44c8-acac-cf3af5b59c8a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.050280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ff76-7321-44c8-acac-cf3af5b59c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.084257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e528ff76-7321-44c8-acac-cf3af5b59c8a None None] MFA Rules not processed for user `62742a74bc6c47d386cc9a6f460ce832`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:42.085566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e528ff76-7321-44c8-acac-cf3af5b59c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:42.159190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 997/1999] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:42.163955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:42.211443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:42.226743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c735d2-b774-4f13-89c2-d950064085e6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:42.232203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:42.232806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.233084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.233352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.233983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.235903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.265448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1003/2000] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:42 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:42.269756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ff76-7321-44c8-acac-cf3af5b59c8a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:42.280658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ff76-7321-44c8-acac-cf3af5b59c8a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:42.289660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:42.339538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:42.340268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.340550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.340821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.341455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.343406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.367672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 998/2001] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:42 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:42.373448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:42.388000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e29e15-db4e-4476-b783-8a19ee8dfdb2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:42.393101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:42.393711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.393993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.394264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.394912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.396544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.407182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1004/2002] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:42.411354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:42.421876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72971b48-325b-43b8-9816-fe8f43757665 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:42.426763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:42.427412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.427686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.427988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] PATH_INFO: `/v3/projects/a352b0dce2ce4cefa29c1c860ac78c7f/users/e09f828dbaf848c78d70ab7c0fda4a0e/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.443710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a352b0dce2ce4cefa29c1c860ac78c7f, user_id=e09f828dbaf848c78d70ab7c0fda4a0e, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.446591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.461795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 999/2003] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:42 2022] PUT /identity/v3/projects/a352b0dce2ce4cefa29c1c860ac78c7f/users/e09f828dbaf848c78d70ab7c0fda4a0e/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:42.466114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:42.476666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8036d-59fc-47c5-a974-d5a11ee45b60 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:42.538961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186f11f-e382-4cd3-9b90-a011d3f47ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:42.538961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186f11f-e382-4cd3-9b90-a011d3f47ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.538961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186f11f-e382-4cd3-9b90-a011d3f47ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.538961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186f11f-e382-4cd3-9b90-a011d3f47ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.539343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186f11f-e382-4cd3-9b90-a011d3f47ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.540013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186f11f-e382-4cd3-9b90-a011d3f47ad8 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.550753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1005/2004] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:42.555013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:42.565441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65146-8261-464e-8706-9e1a544ad42f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:42.570033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:42.570690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.570971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.571251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] PATH_INFO: `/v3/projects/a352b0dce2ce4cefa29c1c860ac78c7f/users/e09f828dbaf848c78d70ab7c0fda4a0e/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.573231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a352b0dce2ce4cefa29c1c860ac78c7f, user_id=e09f828dbaf848c78d70ab7c0fda4a0e, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.575868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.586708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1000/2005] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:42 2022] PUT /identity/v3/projects/a352b0dce2ce4cefa29c1c860ac78c7f/users/e09f828dbaf848c78d70ab7c0fda4a0e/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:42.601505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a0dd9-2353-44a0-8c35-ff8b780ede6a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.601795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a0dd9-2353-44a0-8c35-ff8b780ede6a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.602068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a0dd9-2353-44a0-8c35-ff8b780ede6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.634372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-603a0dd9-2353-44a0-8c35-ff8b780ede6a None None] MFA Rules not processed for user `e09f828dbaf848c78d70ab7c0fda4a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:42.635738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-603a0dd9-2353-44a0-8c35-ff8b780ede6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:42.715473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1006/2006] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:42.726221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:42.727931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:42.786994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532f6f-0b6c-4f29-9740-2bc2b01d0be1 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:42.790863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ed291-9c74-4fcf-b887-0ac106abddac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:42.791499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ed291-9c74-4fcf-b887-0ac106abddac service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:42.791772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ed291-9c74-4fcf-b887-0ac106abddac service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:42.792071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ed291-9c74-4fcf-b887-0ac106abddac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:42.801794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ed291-9c74-4fcf-b887-0ac106abddac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:42.803219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ed291-9c74-4fcf-b887-0ac106abddac service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:42.827922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1001/2007] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:43.458438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a0dd9-2353-44a0-8c35-ff8b780ede6a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:43.507125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:43.526277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a0dd9-2353-44a0-8c35-ff8b780ede6a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:43.530946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:43.531625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:43.531930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:43.532210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] PATH_INFO: `/v3/users/62742a74bc6c47d386cc9a6f460ce832` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:43.533358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62742a74bc6c47d386cc9a6f460ce832)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:43.535285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:43.586467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62742a74bc6c47d386cc9a6f460ce832'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:43.587217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62742a74bc6c47d386cc9a6f460ce832'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:43.587994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62742a74bc6c47d386cc9a6f460ce832'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:43.595726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62742a74bc6c47d386cc9a6f460ce832'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:43.608072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62742a74bc6c47d386cc9a6f460ce832'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:43.628073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1007/2008] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:43 2022] DELETE /identity/v3/users/62742a74bc6c47d386cc9a6f460ce832 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:43.633070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:43.633353 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:43.633620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:43.653230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:43.654440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:43.724833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1002/2009] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:43.735173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:43.736614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:43.798039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79cfed-f84d-4977-8672-d90a39435c54 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:43.802154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:43.802757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:43.803036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:43.803306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:43.812817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:43.814272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:43.824414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1008/2010] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:44.146914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:44.148462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:44.161209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806159bb-1cda-41e9-9c6c-03e8cb9cd7f6 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:44.165923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:44.166570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:44.166846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:44.167115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] PATH_INFO: `/v3/projects/e011aab360e94dee8dcafc96d73811e7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:44.168370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e011aab360e94dee8dcafc96d73811e7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:44.171022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:44.173989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:44.224143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e011aab360e94dee8dcafc96d73811e7', 'request_id': 'req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.230946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e011aab360e94dee8dcafc96d73811e7', 'request_id': 'req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.236608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e011aab360e94dee8dcafc96d73811e7', 'request_id': 'req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.250371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] The token cache is being invalidate because project e011aab360e94dee8dcafc96d73811e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:44.250591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.251191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.252717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1003/2011] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:44 2022] DELETE /identity/v3/projects/e011aab360e94dee8dcafc96d73811e7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:44.256722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:44.266251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c719b-7fbf-47b9-acae-e1dbf0ccf55d service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:44.325815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:44.326452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:44.326761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:44.327033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] PATH_INFO: `/v3/users/e09f828dbaf848c78d70ab7c0fda4a0e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:44.328243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09f828dbaf848c78d70ab7c0fda4a0e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:44.330184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:44.334770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:44.379183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09f828dbaf848c78d70ab7c0fda4a0e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.379829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09f828dbaf848c78d70ab7c0fda4a0e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.380458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09f828dbaf848c78d70ab7c0fda4a0e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.387291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09f828dbaf848c78d70ab7c0fda4a0e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.399457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ba32d6b2-a896-4c1c-bca5-a15720e856b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09f828dbaf848c78d70ab7c0fda4a0e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.415281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1009/2012] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:44 2022] DELETE /identity/v3/users/e09f828dbaf848c78d70ab7c0fda4a0e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:44.688816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:44.690310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:44.708610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4462ba7-3e64-4f28-8abb-3158cfc0d76c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:44.764408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:44.764953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:44.765223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:44.765492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] PATH_INFO: `/v3/projects/a352b0dce2ce4cefa29c1c860ac78c7f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:44.766606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a352b0dce2ce4cefa29c1c860ac78c7f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:44.768584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:44.819054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a352b0dce2ce4cefa29c1c860ac78c7f', 'request_id': 'req-562c3e66-3e7d-414e-8487-c53c9b5d9835'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.825175 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a352b0dce2ce4cefa29c1c860ac78c7f', 'request_id': 'req-562c3e66-3e7d-414e-8487-c53c9b5d9835'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.831134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a352b0dce2ce4cefa29c1c860ac78c7f', 'request_id': 'req-562c3e66-3e7d-414e-8487-c53c9b5d9835'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.846221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] The token cache is being invalidate because project a352b0dce2ce4cefa29c1c860ac78c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:44.846506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.847136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-562c3e66-3e7d-414e-8487-c53c9b5d9835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:44.848687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1004/2013] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:44 2022] DELETE /identity/v3/projects/a352b0dce2ce4cefa29c1c860ac78c7f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:44.853997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426c292-5aac-41e0-8acd-77564313bebb None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:44.854491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426c292-5aac-41e0-8acd-77564313bebb None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:44.854590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426c292-5aac-41e0-8acd-77564313bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:44.874794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3426c292-5aac-41e0-8acd-77564313bebb None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:44.876000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3426c292-5aac-41e0-8acd-77564313bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:44.936894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1010/2014] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:44.944772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd94577-bca7-4500-96a7-0e7b3bf0d32e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:44.945172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd94577-bca7-4500-96a7-0e7b3bf0d32e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:44.945508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd94577-bca7-4500-96a7-0e7b3bf0d32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:44.966244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-edd94577-bca7-4500-96a7-0e7b3bf0d32e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:44.969768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-edd94577-bca7-4500-96a7-0e7b3bf0d32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:44.980849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1005/2015] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:44.985320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426c292-5aac-41e0-8acd-77564313bebb None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:44.995542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426c292-5aac-41e0-8acd-77564313bebb None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.000251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d68053-4719-44b3-9569-24501357068b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.000855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d68053-4719-44b3-9569-24501357068b admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.001130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d68053-4719-44b3-9569-24501357068b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.001398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d68053-4719-44b3-9569-24501357068b admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.002046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d68053-4719-44b3-9569-24501357068b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:45.002349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d68053-4719-44b3-9569-24501357068b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.004100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d68053-4719-44b3-9569-24501357068b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.016269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1011/2016] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:45.020845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd94577-bca7-4500-96a7-0e7b3bf0d32e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.030412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd94577-bca7-4500-96a7-0e7b3bf0d32e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.036695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.037345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.037558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.037791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.038449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.040367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.056413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:45.073826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1006/2017] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:45 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:45.079488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d68053-4719-44b3-9569-24501357068b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.090527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d68053-4719-44b3-9569-24501357068b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.159982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.160720 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.161004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.161282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.161935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.163915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.183862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:45.192701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1012/2018] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:45 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:45.194117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.204816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c7078-d564-4876-b487-4e5884f84b66 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.209339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.209945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.210222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.210492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.211135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.212836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.223170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1007/2019] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:45.227201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.238383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b61051-4e27-4822-b0f2-576cd129edc5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.243114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12319c88-ec9f-422d-a93b-c96bf77c8d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.243745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319c88-ec9f-422d-a93b-c96bf77c8d8d admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.244143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319c88-ec9f-422d-a93b-c96bf77c8d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.244415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319c88-ec9f-422d-a93b-c96bf77c8d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.245067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12319c88-ec9f-422d-a93b-c96bf77c8d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.246701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12319c88-ec9f-422d-a93b-c96bf77c8d8d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.257653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1013/2020] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:45.261918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.272288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916997e-654d-4470-b3b9-a11bdb11842e admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.281702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.283965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.284610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.293929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] PATH_INFO: `/v3/projects/07c537fe35db439c89addc27dc5046eb/users/92b64e9d1f15474195f72e079a09efe5/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.313813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c537fe35db439c89addc27dc5046eb, user_id=92b64e9d1f15474195f72e079a09efe5, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.317732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.339978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1008/2021] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:45 2022] PUT /identity/v3/projects/07c537fe35db439c89addc27dc5046eb/users/92b64e9d1f15474195f72e079a09efe5/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:45.350128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5ba3a-4bd2-41b3-b8ae-6ffb2a2a389a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.350420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5ba3a-4bd2-41b3-b8ae-6ffb2a2a389a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.350692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5ba3a-4bd2-41b3-b8ae-6ffb2a2a389a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.379710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b0c5ba3a-4bd2-41b3-b8ae-6ffb2a2a389a None None] MFA Rules not processed for user `92b64e9d1f15474195f72e079a09efe5`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:45.380931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b0c5ba3a-4bd2-41b3-b8ae-6ffb2a2a389a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:45.454186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1014/2022] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:45.458895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.521384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fe3e-7794-4eed-a738-cacd3312ab44 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.526420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.527029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.527303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.527556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.528250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.530143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.560760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1009/2023] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:45 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:45.564831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5ba3a-4bd2-41b3-b8ae-6ffb2a2a389a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.575471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5ba3a-4bd2-41b3-b8ae-6ffb2a2a389a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.631481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.632129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.632403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.632675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.633319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.635260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.660502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1015/2024] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:45 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:52:45.664581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.674802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ec078-267f-462c-9995-eb4593edf014 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.679650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.680282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.680565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.680832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.681519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.683110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.694060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1010/2025] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:45.703018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.708223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af811978-267f-4c96-9f72-597c057e204e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.713016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.713670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.713948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.714220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] PATH_INFO: `/v3/projects/f2f427f9dc664b29a61d9a7423acc392/users/1e7d4989b86c431f8b704ddab3a0c771/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.728849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f427f9dc664b29a61d9a7423acc392, user_id=1e7d4989b86c431f8b704ddab3a0c771, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.731469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.746380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1016/2026] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:45 2022] PUT /identity/v3/projects/f2f427f9dc664b29a61d9a7423acc392/users/1e7d4989b86c431f8b704ddab3a0c771/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:45.750441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.761004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd25e48-f3b8-4747-a247-300b9c46e9cb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.819637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95329760-80c9-49ef-90af-b14a419b6e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.820310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329760-80c9-49ef-90af-b14a419b6e40 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.820593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329760-80c9-49ef-90af-b14a419b6e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.820867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95329760-80c9-49ef-90af-b14a419b6e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.821656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95329760-80c9-49ef-90af-b14a419b6e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.823272 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95329760-80c9-49ef-90af-b14a419b6e40 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.833549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1011/2027] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:52:45.838253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.849081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d30b1-26c8-4290-a807-b078bd3c96df admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:45.853957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:45.854615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.855202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.855202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] PATH_INFO: `/v3/projects/f2f427f9dc664b29a61d9a7423acc392/users/1e7d4989b86c431f8b704ddab3a0c771/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.857293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f427f9dc664b29a61d9a7423acc392, user_id=1e7d4989b86c431f8b704ddab3a0c771, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:45.859967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:45.871734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1017/2028] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:45 2022] PUT /identity/v3/projects/f2f427f9dc664b29a61d9a7423acc392/users/1e7d4989b86c431f8b704ddab3a0c771/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:45.882122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:45.882405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:45.882676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:45.913342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] MFA Rules not processed for user `1e7d4989b86c431f8b704ddab3a0c771`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:45.914557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:45.931257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:45.982552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1012/2029] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:45.994330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:45.996164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:46.031441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:46.061115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de5226-d6eb-4733-b128-40dc3ddaadc5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:46.065951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4dc7e-96f2-4a73-a333-e33fd51c3868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:46.066642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4dc7e-96f2-4a73-a333-e33fd51c3868 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:46.067042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4dc7e-96f2-4a73-a333-e33fd51c3868 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:46.067382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4dc7e-96f2-4a73-a333-e33fd51c3868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:46.127234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4dc7e-96f2-4a73-a333-e33fd51c3868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:46.128705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4dc7e-96f2-4a73-a333-e33fd51c3868 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:46.138064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1018/2030] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:48.923615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:48.925120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:49.022364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cbefb-3b11-4134-9075-04210f18b01c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:49.026879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:49.027527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:49.027803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:49.028085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] PATH_INFO: `/v3/users/cec4a3cc81a449a9adf2d73ea454499d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:49.029875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec4a3cc81a449a9adf2d73ea454499d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:49.031555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:49.083950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec4a3cc81a449a9adf2d73ea454499d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.084686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec4a3cc81a449a9adf2d73ea454499d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.093443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec4a3cc81a449a9adf2d73ea454499d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.094148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec4a3cc81a449a9adf2d73ea454499d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.110975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec4a3cc81a449a9adf2d73ea454499d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.131358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1013/2031] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:48 2022] DELETE /identity/v3/users/cec4a3cc81a449a9adf2d73ea454499d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:49.133636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:49.134714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:49.135289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:49.157756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:49.159448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:49.232739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1019/2032] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:49.239650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:49.241083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:49.275928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:49.299111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff368d1c-b971-44a9-9bd3-9c52e39f3311 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:49.302668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:49.303278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:49.303551 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:49.303818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:49.312886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:49.314311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:49.323720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1014/2033] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:49.588384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:49.590047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:49.602492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c0e48-becb-4244-9a2e-5ef0ba3b925c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:49.608060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:49.608711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:49.608985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:49.609515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] PATH_INFO: `/v3/projects/2a22714b2eef478f967f243e351dbb1b` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:49.610752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a22714b2eef478f967f243e351dbb1b)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:49.613077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:49.663414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a22714b2eef478f967f243e351dbb1b', 'request_id': 'req-1503a337-512f-46fd-951d-7beb4b3559ad'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.670131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a22714b2eef478f967f243e351dbb1b', 'request_id': 'req-1503a337-512f-46fd-951d-7beb4b3559ad'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.676014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a22714b2eef478f967f243e351dbb1b', 'request_id': 'req-1503a337-512f-46fd-951d-7beb4b3559ad'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.690738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] The token cache is being invalidate because project 2a22714b2eef478f967f243e351dbb1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:49.691024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.691722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.693342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1020/2034] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:49 2022] DELETE /identity/v3/projects/2a22714b2eef478f967f243e351dbb1b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:49.697750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:49.707794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a9743-b619-4a12-8bc5-870a9f9c8ae4 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:49.769803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:49.770447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:49.770835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:49.771108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] PATH_INFO: `/v3/users/1097bf0fd7704e7c93fba36aca0b6229` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:49.772309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1097bf0fd7704e7c93fba36aca0b6229)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:49.774422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:49.824534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1097bf0fd7704e7c93fba36aca0b6229'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.825200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1097bf0fd7704e7c93fba36aca0b6229'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.832240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1097bf0fd7704e7c93fba36aca0b6229'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.832855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1097bf0fd7704e7c93fba36aca0b6229'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.847860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1097bf0fd7704e7c93fba36aca0b6229'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:49.861912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1015/2035] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:49 2022] DELETE /identity/v3/users/1097bf0fd7704e7c93fba36aca0b6229 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:50.221511 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:50.223052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:50.237934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503a337-512f-46fd-951d-7beb4b3559ad admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:50.243765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:50.308107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:50.308835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:50.309195 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:50.309529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] PATH_INFO: `/v3/projects/121ab10fdfe54853bcffa73f57fc0b40` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:50.310699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=121ab10fdfe54853bcffa73f57fc0b40)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:50.312697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:50.367245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121ab10fdfe54853bcffa73f57fc0b40', 'request_id': 'req-8029ee7b-5b08-4964-9742-a0a85532c74b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.374220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121ab10fdfe54853bcffa73f57fc0b40', 'request_id': 'req-8029ee7b-5b08-4964-9742-a0a85532c74b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.380214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121ab10fdfe54853bcffa73f57fc0b40', 'request_id': 'req-8029ee7b-5b08-4964-9742-a0a85532c74b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.397068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] The token cache is being invalidate because project 121ab10fdfe54853bcffa73f57fc0b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:50.397385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.397967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.399520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1021/2036] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:50 2022] DELETE /identity/v3/projects/121ab10fdfe54853bcffa73f57fc0b40 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:50.403564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:50.405088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:50.417563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f69d8-175c-4126-bb19-1d0f187daf7e admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:50.478107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:50.486296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:50.487395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:50.487702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:50.488058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] PATH_INFO: `/v3/users/455b53300f334ee8aa733d574c241052` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:50.489508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=455b53300f334ee8aa733d574c241052)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:50.491783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:50.544028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455b53300f334ee8aa733d574c241052'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.544693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455b53300f334ee8aa733d574c241052'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.551525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455b53300f334ee8aa733d574c241052'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.552182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455b53300f334ee8aa733d574c241052'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.563905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9d95512c-d7d4-406b-83f3-f86b18621698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455b53300f334ee8aa733d574c241052'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:50.579102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1016/2037] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:52:50 2022] DELETE /identity/v3/users/455b53300f334ee8aa733d574c241052 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:50.857210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:50.858912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:50.874534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029ee7b-5b08-4964-9742-a0a85532c74b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:50.933282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:50.933916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:50.934190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:50.934456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] PATH_INFO: `/v3/projects/f64336ca2b374f8a803fbcff46a8d18f` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:50.935565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64336ca2b374f8a803fbcff46a8d18f)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:50.937500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:50.987425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:50.994444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64336ca2b374f8a803fbcff46a8d18f', 'request_id': 'req-f2d82ead-1213-450b-a103-9ebd1bb4093f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:51.001745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64336ca2b374f8a803fbcff46a8d18f', 'request_id': 'req-f2d82ead-1213-450b-a103-9ebd1bb4093f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:51.007614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64336ca2b374f8a803fbcff46a8d18f', 'request_id': 'req-f2d82ead-1213-450b-a103-9ebd1bb4093f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:51.021975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] The token cache is being invalidate because project f64336ca2b374f8a803fbcff46a8d18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:52:51.022263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:51.022876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f2d82ead-1213-450b-a103-9ebd1bb4093f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:52:51.024422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1022/2038] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:52:50 2022] DELETE /identity/v3/projects/f64336ca2b374f8a803fbcff46a8d18f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:52:51.029568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e631c-dd7b-4249-a377-09a09811bc69 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.029851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e631c-dd7b-4249-a377-09a09811bc69 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.030145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e631c-dd7b-4249-a377-09a09811bc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.049908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d48e631c-dd7b-4249-a377-09a09811bc69 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:51.051125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d48e631c-dd7b-4249-a377-09a09811bc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:51.114680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1017/2039] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:52:51.122916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de63df5-5cd5-4acb-81b8-cf6d71d90edd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.123199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de63df5-5cd5-4acb-81b8-cf6d71d90edd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.123471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de63df5-5cd5-4acb-81b8-cf6d71d90edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.145756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0de63df5-5cd5-4acb-81b8-cf6d71d90edd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:51.146986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0de63df5-5cd5-4acb-81b8-cf6d71d90edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:51.156887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1023/2040] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:51.161045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e631c-dd7b-4249-a377-09a09811bc69 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:51.170633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e631c-dd7b-4249-a377-09a09811bc69 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:51.175670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:51.176333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.176614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.176884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.177634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:52:51.177941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:51.179535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:51.191371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1018/2041] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:52:51.195814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de63df5-5cd5-4acb-81b8-cf6d71d90edd None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:51.205801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de63df5-5cd5-4acb-81b8-cf6d71d90edd None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:51.211083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:51.211688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.212114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.212389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.213042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:51.214966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:51.245377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1024/2042] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:52:51 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:51.249491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:51.259406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c269-7822-40e1-981c-a970b39b0351 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:51.323146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:51.323808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.324135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.324409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.325047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:51.326962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:51.336341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:51.361081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1019/2043] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:52:51 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:52:51.365777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:51.375909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643c9dd-d541-4a5f-b413-3fab2c4d2ff5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:51.380644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:51.381367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.381633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.381901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.382542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:51.384142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:51.394307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1025/2044] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:51.398305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:51.408107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd290002-9a4e-4a3a-9c57-fa7208b2d76b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:51.413544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418410b3-1661-4df8-95de-1a95ae9ed479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:51.414152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418410b3-1661-4df8-95de-1a95ae9ed479 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.414426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418410b3-1661-4df8-95de-1a95ae9ed479 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.414727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418410b3-1661-4df8-95de-1a95ae9ed479 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.415449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418410b3-1661-4df8-95de-1a95ae9ed479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:51.417362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418410b3-1661-4df8-95de-1a95ae9ed479 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:51.428759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1020/2045] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:52:51.433057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:51.442978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5a995-00d7-460c-b95f-8122c5ae197c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:51.447982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:51.448131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.448467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.448740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] PATH_INFO: `/v3/projects/aa07ba2c096a401f9c72739e7e2ee18e/users/3bebc9b1b9c0425ea0032575a509a88f/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.464632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa07ba2c096a401f9c72739e7e2ee18e, user_id=3bebc9b1b9c0425ea0032575a509a88f, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:51.467252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:51.481914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1026/2046] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:52:51 2022] PUT /identity/v3/projects/aa07ba2c096a401f9c72739e7e2ee18e/users/3bebc9b1b9c0425ea0032575a509a88f/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:52:51.493707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.494017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.494285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.523091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] MFA Rules not processed for user `3bebc9b1b9c0425ea0032575a509a88f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:51.524338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:51.589153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1021/2047] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:52:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:51.600571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:51.602298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:51.671118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fc815-33e5-4523-81a3-38867f2a83b4 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:51.674692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1373d8-9f66-49de-a92a-a00ebe722e3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:51.675298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1373d8-9f66-49de-a92a-a00ebe722e3c service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:51.675576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1373d8-9f66-49de-a92a-a00ebe722e3c service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:51.675852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1373d8-9f66-49de-a92a-a00ebe722e3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:51.683959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1373d8-9f66-49de-a92a-a00ebe722e3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:51.685386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1373d8-9f66-49de-a92a-a00ebe722e3c service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:51.693946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1027/2048] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:01.586131 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:01.690066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:04.172779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:04.174587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:04.241924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6d6f1-718a-46db-a221-469025c64dd4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:04.247140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:04.247837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:04.248138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:04.248414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] PATH_INFO: `/v3/users/b0e38b63fb94447d86997507f9f2e4e3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:04.249564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0e38b63fb94447d86997507f9f2e4e3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:04.251483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:04.311950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0e38b63fb94447d86997507f9f2e4e3'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:04.312634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0e38b63fb94447d86997507f9f2e4e3'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:04.320201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0e38b63fb94447d86997507f9f2e4e3'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:04.320933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0e38b63fb94447d86997507f9f2e4e3'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:04.334718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0e38b63fb94447d86997507f9f2e4e3'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:04.349780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1022/2049] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:04 2022] DELETE /identity/v3/users/b0e38b63fb94447d86997507f9f2e4e3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:04.354512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:04.354972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:04.355060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:04.374492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:04.375679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:04.445746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1028/2050] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:04.452719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:04.454247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:04.520423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbe33d-024e-475b-9bbd-c8fe4614c013 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:04.525420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:04.526039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:04.526319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:04.526592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:04.536921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:04.538374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:04.548003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1023/2051] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:04.654847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:04.656642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:04.718279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1c741-9d1a-4615-9120-19ce73472ebc None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:04.722003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:04.722610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:04.722884 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:04.723149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:04.725248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:04.799902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:04.802168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:04.815672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1029/2052] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:04.942048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:04.943516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:04.957935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02334297-1f7c-4e08-bbe8-ab6a220ef477 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:04.964927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:04.966642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:04.966642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:04.966642 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] PATH_INFO: `/v3/projects/f0bbff82c56e4c84874a4d8fc964af00` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:04.974167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0bbff82c56e4c84874a4d8fc964af00)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:04.976206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:05.004771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:05.056277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0bbff82c56e4c84874a4d8fc964af00', 'request_id': 'req-43412aa9-2d45-4263-8f7d-d32381c574c0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.059532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0bbff82c56e4c84874a4d8fc964af00', 'request_id': 'req-43412aa9-2d45-4263-8f7d-d32381c574c0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.064916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0bbff82c56e4c84874a4d8fc964af00', 'request_id': 'req-43412aa9-2d45-4263-8f7d-d32381c574c0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.089371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] The token cache is being invalidate because project f0bbff82c56e4c84874a4d8fc964af00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:05.089666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.091411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.093397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1024/2053] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:04 2022] DELETE /identity/v3/projects/f0bbff82c56e4c84874a4d8fc964af00 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:05.097430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:05.123828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b87379-d37b-4ac8-842c-0adcc332f1bb service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:05.232186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:05.244021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:05.244981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:05.245461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:05.245957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] PATH_INFO: `/v3/users/d8f73b49a8e8482c9dbec8fcec09b3b0` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:05.247284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f73b49a8e8482c9dbec8fcec09b3b0)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:05.249630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:05.308652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f73b49a8e8482c9dbec8fcec09b3b0'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.309305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f73b49a8e8482c9dbec8fcec09b3b0'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.309923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f73b49a8e8482c9dbec8fcec09b3b0'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.317096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f73b49a8e8482c9dbec8fcec09b3b0'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.329449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-5424f275-208f-43b7-aabf-13329f9f8a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f73b49a8e8482c9dbec8fcec09b3b0'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.344616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1030/2054] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:05 2022] DELETE /identity/v3/users/d8f73b49a8e8482c9dbec8fcec09b3b0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:05.624808 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:05.626436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:05.641447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43412aa9-2d45-4263-8f7d-d32381c574c0 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:05.709389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:05.710142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:05.710509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:05.710983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] PATH_INFO: `/v3/projects/4a9a16c93b6f4662bf2303884bd0fb71` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:05.712400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9a16c93b6f4662bf2303884bd0fb71)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:05.715104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:05.771093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9a16c93b6f4662bf2303884bd0fb71', 'request_id': 'req-10e4ec7e-e473-4927-9427-bb98caa38b08'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.777540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9a16c93b6f4662bf2303884bd0fb71', 'request_id': 'req-10e4ec7e-e473-4927-9427-bb98caa38b08'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.783680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9a16c93b6f4662bf2303884bd0fb71', 'request_id': 'req-10e4ec7e-e473-4927-9427-bb98caa38b08'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.798384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] The token cache is being invalidate because project 4a9a16c93b6f4662bf2303884bd0fb71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:05.798693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.799289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-10e4ec7e-e473-4927-9427-bb98caa38b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:05.801147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1025/2055] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:05 2022] DELETE /identity/v3/projects/4a9a16c93b6f4662bf2303884bd0fb71 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:05.806758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ad684-3f80-4706-9987-4b1a0fcc85a5 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:05.807267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ad684-3f80-4706-9987-4b1a0fcc85a5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:05.807411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ad684-3f80-4706-9987-4b1a0fcc85a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:05.829789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d66ad684-3f80-4706-9987-4b1a0fcc85a5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:05.831098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d66ad684-3f80-4706-9987-4b1a0fcc85a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:05.893322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1031/2056] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:05.901555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfaf87-7e0e-4bbe-a4a8-a6805444fcbb None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:05.901925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfaf87-7e0e-4bbe-a4a8-a6805444fcbb None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:05.902107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfaf87-7e0e-4bbe-a4a8-a6805444fcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:05.925254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-45cfaf87-7e0e-4bbe-a4a8-a6805444fcbb None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:05.926490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-45cfaf87-7e0e-4bbe-a4a8-a6805444fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:05.936932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1026/2057] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:05.941987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ad684-3f80-4706-9987-4b1a0fcc85a5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:05.954354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ad684-3f80-4706-9987-4b1a0fcc85a5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:05.962917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:05.963558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:05.963853 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:05.964137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:05.964801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:05.965273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:05.967013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:05.980341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1032/2058] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:05.985317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfaf87-7e0e-4bbe-a4a8-a6805444fcbb None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:05.995153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfaf87-7e0e-4bbe-a4a8-a6805444fcbb None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.000230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.000842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.001239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.001520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.002197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.004161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.037630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1027/2059] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:05 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:06.038941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.055480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610629ec-294e-4054-ad88-97844c773bcf admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.111378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:06.118380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.119497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.119497 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.119688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.120345 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.122843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.161075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1033/2060] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:06 2022] POST /identity/v3/users => generated 282 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:06.161639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.172781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bb5d2-5721-4239-b759-f9550157666c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.190409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.191419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.191815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.192208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.193138 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.195236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.198450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:06.213998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1028/2061] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:53:06.214647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.227681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43551f54-f6b3-4a6c-a29f-b72755b37efe admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.242683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e938b-8fcf-426e-b349-bed60726fd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.243397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e938b-8fcf-426e-b349-bed60726fd3a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.243751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e938b-8fcf-426e-b349-bed60726fd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.244155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e938b-8fcf-426e-b349-bed60726fd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.244883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e938b-8fcf-426e-b349-bed60726fd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.246620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e938b-8fcf-426e-b349-bed60726fd3a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.263652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1034/2062] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:53:06.264865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.275090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80034599-1704-4d21-ab5d-865e7d5aec3f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.281074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.282100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.282643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.283048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] PATH_INFO: `/v3/projects/210e6ecf7861431094e27745b61245b1/users/7b3dd21c06ee404ca2cf7e2e4700e1db/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.300778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=210e6ecf7861431094e27745b61245b1, user_id=7b3dd21c06ee404ca2cf7e2e4700e1db, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.303615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.317962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1029/2063] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:06 2022] PUT /identity/v3/projects/210e6ecf7861431094e27745b61245b1/users/7b3dd21c06ee404ca2cf7e2e4700e1db/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:06.328781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2948e50-2027-4c21-a918-6dc9d7916152 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.329142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2948e50-2027-4c21-a918-6dc9d7916152 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.329487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2948e50-2027-4c21-a918-6dc9d7916152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.358472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c2948e50-2027-4c21-a918-6dc9d7916152 None None] MFA Rules not processed for user `7b3dd21c06ee404ca2cf7e2e4700e1db`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:06.359833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c2948e50-2027-4c21-a918-6dc9d7916152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:06.436862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1035/2064] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:06.441472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.509973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab6f6-f99b-4a0c-a481-4d51f84d1e0c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.518683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.519433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.519714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.520015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.520770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.522713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.553629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1030/2065] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:06 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:06.557447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2948e50-2027-4c21-a918-6dc9d7916152 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.570385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2948e50-2027-4c21-a918-6dc9d7916152 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.631568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.632203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.632481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.632899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.633385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.635646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.674796 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1036/2066] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:06 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:06.678570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.689677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8f38a-1901-41d2-bd3e-37e0f2c8d1cd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.694602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.695208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.695484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.695750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.696423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.698007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.709001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1031/2067] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:06.713099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.723397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb24e5-27f5-4912-bf33-17478d312dc9 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.729171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.729821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.730106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.730356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] PATH_INFO: `/v3/projects/3658a1739b67408fa45c155c2b430f59/users/128e24d1d71644bd87203c5f4f3e4c44/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.751069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3658a1739b67408fa45c155c2b430f59, user_id=128e24d1d71644bd87203c5f4f3e4c44, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.753817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.770135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1037/2068] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:06 2022] PUT /identity/v3/projects/3658a1739b67408fa45c155c2b430f59/users/128e24d1d71644bd87203c5f4f3e4c44/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:06.774150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.786300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677109a-7e35-4861-9119-08aed917fd2a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.845970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508e651-d1a3-4294-b4fe-e0772299c5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.846493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508e651-d1a3-4294-b4fe-e0772299c5dc admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.846766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508e651-d1a3-4294-b4fe-e0772299c5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.847030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508e651-d1a3-4294-b4fe-e0772299c5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.847669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508e651-d1a3-4294-b4fe-e0772299c5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.849294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508e651-d1a3-4294-b4fe-e0772299c5dc admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.859057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1032/2069] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:06.863669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:06.875132 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8159f-d2c4-44c7-b55f-ec12abcc2af1 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:06.881098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:06.881818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.882073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.882369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] PATH_INFO: `/v3/projects/3658a1739b67408fa45c155c2b430f59/users/128e24d1d71644bd87203c5f4f3e4c44/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.884597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3658a1739b67408fa45c155c2b430f59, user_id=128e24d1d71644bd87203c5f4f3e4c44, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:06.887235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:06.899148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1038/2070] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:06 2022] PUT /identity/v3/projects/3658a1739b67408fa45c155c2b430f59/users/128e24d1d71644bd87203c5f4f3e4c44/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:06.910725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cda3b6-fcac-4ddf-93d2-68becf7432f4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:06.910998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cda3b6-fcac-4ddf-93d2-68becf7432f4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:06.911281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cda3b6-fcac-4ddf-93d2-68becf7432f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:06.941730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e0cda3b6-fcac-4ddf-93d2-68becf7432f4 None None] MFA Rules not processed for user `128e24d1d71644bd87203c5f4f3e4c44`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:06.942948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e0cda3b6-fcac-4ddf-93d2-68becf7432f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:06.979326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:07.022907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1033/2071] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:07.034830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:07.036605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:07.057950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:07.097768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b67b2-a64b-48c8-88d6-bf8762af0938 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:07.101251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0adaf-c788-4d5f-ad05-42a600c1703e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:07.101882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0adaf-c788-4d5f-ad05-42a600c1703e service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:07.102156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0adaf-c788-4d5f-ad05-42a600c1703e service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:07.102426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0adaf-c788-4d5f-ad05-42a600c1703e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:07.111256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0adaf-c788-4d5f-ad05-42a600c1703e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:07.113040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0adaf-c788-4d5f-ad05-42a600c1703e service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:07.123210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1039/2072] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:07.611826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cda3b6-fcac-4ddf-93d2-68becf7432f4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:07.690228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cda3b6-fcac-4ddf-93d2-68becf7432f4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:07.695172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:07.695815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:07.696129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:07.696400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] PATH_INFO: `/v3/users/7b3dd21c06ee404ca2cf7e2e4700e1db` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:07.697561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3dd21c06ee404ca2cf7e2e4700e1db)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:07.699525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:07.750804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3dd21c06ee404ca2cf7e2e4700e1db'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:07.751555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3dd21c06ee404ca2cf7e2e4700e1db'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:07.758990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3dd21c06ee404ca2cf7e2e4700e1db'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:07.759768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3dd21c06ee404ca2cf7e2e4700e1db'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:07.774496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3dd21c06ee404ca2cf7e2e4700e1db'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:07.788787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1034/2073] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:07 2022] DELETE /identity/v3/users/7b3dd21c06ee404ca2cf7e2e4700e1db => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:07.793657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:07.793949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:07.794260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:07.816114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:07.817328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:07.880872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1040/2074] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:07.890817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:07.892278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:07.946478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:07.953168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d8a33-c9dd-462c-a04d-80cfa67544cb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:07.958595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:07.959313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:07.959878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:07.960362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:07.976253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:07.977726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:07.988790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1035/2075] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:08.285088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:08.286538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:08.300835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562049a7-be76-4642-9ec4-8e5696957029 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:08.308949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:08.309584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:08.310636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:08.311382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] PATH_INFO: `/v3/projects/210e6ecf7861431094e27745b61245b1` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:08.312581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=210e6ecf7861431094e27745b61245b1)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:08.314467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:08.386880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210e6ecf7861431094e27745b61245b1', 'request_id': 'req-241548e8-8eb4-4339-8ed4-a7a358484d11'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.394202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210e6ecf7861431094e27745b61245b1', 'request_id': 'req-241548e8-8eb4-4339-8ed4-a7a358484d11'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.402849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210e6ecf7861431094e27745b61245b1', 'request_id': 'req-241548e8-8eb4-4339-8ed4-a7a358484d11'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.416184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] The token cache is being invalidate because project 210e6ecf7861431094e27745b61245b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:08.416471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.417085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.418583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1041/2076] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:08 2022] DELETE /identity/v3/projects/210e6ecf7861431094e27745b61245b1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:08.422794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:08.432876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4073ba-f7b5-4ffb-a9b2-0dc7a1d5f97d service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:08.491037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:08.491673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:08.491976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:08.492245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] PATH_INFO: `/v3/users/128e24d1d71644bd87203c5f4f3e4c44` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:08.493483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=128e24d1d71644bd87203c5f4f3e4c44)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:08.495456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:08.549450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128e24d1d71644bd87203c5f4f3e4c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.550108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128e24d1d71644bd87203c5f4f3e4c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.556994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128e24d1d71644bd87203c5f4f3e4c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.557627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128e24d1d71644bd87203c5f4f3e4c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.569517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0e5d0be5-acb9-4ad9-93f7-9e3fcbbd9a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128e24d1d71644bd87203c5f4f3e4c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:08.583120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1036/2077] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:08 2022] DELETE /identity/v3/users/128e24d1d71644bd87203c5f4f3e4c44 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:08.909981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:08.911553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:08.930879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241548e8-8eb4-4339-8ed4-a7a358484d11 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:08.973684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:08.990342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:08.990979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:08.991255 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:08.991525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] PATH_INFO: `/v3/projects/3658a1739b67408fa45c155c2b430f59` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:08.992661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3658a1739b67408fa45c155c2b430f59)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:08.994580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.041928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3658a1739b67408fa45c155c2b430f59', 'request_id': 'req-792880f5-6352-49da-88f4-15ee8caab855'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:09.047948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3658a1739b67408fa45c155c2b430f59', 'request_id': 'req-792880f5-6352-49da-88f4-15ee8caab855'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:09.053516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3658a1739b67408fa45c155c2b430f59', 'request_id': 'req-792880f5-6352-49da-88f4-15ee8caab855'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:09.067664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] The token cache is being invalidate because project 3658a1739b67408fa45c155c2b430f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:09.067982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:09.068832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-792880f5-6352-49da-88f4-15ee8caab855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:09.070393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1042/2078] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:08 2022] DELETE /identity/v3/projects/3658a1739b67408fa45c155c2b430f59 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:09.075375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed359ff-5e6e-49d3-9e66-cb0504cc75e4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.075924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed359ff-5e6e-49d3-9e66-cb0504cc75e4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.076008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed359ff-5e6e-49d3-9e66-cb0504cc75e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.096458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6ed359ff-5e6e-49d3-9e66-cb0504cc75e4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:09.097859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6ed359ff-5e6e-49d3-9e66-cb0504cc75e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:09.157051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1037/2079] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:09.164840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63587ec-e611-4f8e-8129-6915484c8813 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.165127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63587ec-e611-4f8e-8129-6915484c8813 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.165397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63587ec-e611-4f8e-8129-6915484c8813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.185565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e63587ec-e611-4f8e-8129-6915484c8813 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:09.186772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e63587ec-e611-4f8e-8129-6915484c8813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:09.195923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1043/2080] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:09.200287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed359ff-5e6e-49d3-9e66-cb0504cc75e4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.210529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed359ff-5e6e-49d3-9e66-cb0504cc75e4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.216045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.216648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.216950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.217212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.217853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:09.218158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.219749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.222220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:09.236151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1038/2081] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:09.237694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63587ec-e611-4f8e-8129-6915484c8813 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.247174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63587ec-e611-4f8e-8129-6915484c8813 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.253203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.253830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.254101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.254368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.254976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.256942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.287160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1044/2082] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:09 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:09.291525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.302032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d865-3aa0-490c-a45e-4cb5e5416dab admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.359435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.360039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.360315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.360583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.361222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.363475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.385739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1039/2083] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:09 2022] POST /identity/v3/users => generated 278 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:09.390134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.404621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf3a09-915a-49ea-92ca-c6be3ed01aa0 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.409751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.410317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.410594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.410860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.411506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.413316 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.427125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1045/2084] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:09.428216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.438831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da3c62-9c76-4bd4-a003-01114b1f0767 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.443779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fd08c-7d7d-4fb6-a4f2-3074a1377281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.444410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fd08c-7d7d-4fb6-a4f2-3074a1377281 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.444684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fd08c-7d7d-4fb6-a4f2-3074a1377281 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.444950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fd08c-7d7d-4fb6-a4f2-3074a1377281 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.445592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fd08c-7d7d-4fb6-a4f2-3074a1377281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.447165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fd08c-7d7d-4fb6-a4f2-3074a1377281 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.458094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1040/2085] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:09.462205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.472543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72531724-3079-4f17-beb3-3d4631705f97 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.477683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.478348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.478626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.478898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] PATH_INFO: `/v3/projects/739295c6103a427094026324bdc86f37/users/68a43aad622e429eb504e4c3307a116b/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.495629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=739295c6103a427094026324bdc86f37, user_id=68a43aad622e429eb504e4c3307a116b, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.498849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.516269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1046/2086] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:09 2022] PUT /identity/v3/projects/739295c6103a427094026324bdc86f37/users/68a43aad622e429eb504e4c3307a116b/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:09.528123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c69272-82c6-41d2-a6a2-837ab0289509 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.528398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c69272-82c6-41d2-a6a2-837ab0289509 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.528717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c69272-82c6-41d2-a6a2-837ab0289509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.556733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-15c69272-82c6-41d2-a6a2-837ab0289509 None None] MFA Rules not processed for user `68a43aad622e429eb504e4c3307a116b`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:09.557938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-15c69272-82c6-41d2-a6a2-837ab0289509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:09.622619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1041/2087] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:09.627083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.721045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d5da5-a83a-40a8-b420-778e2218a3b0 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.726129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.726725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.727000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.727276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.727921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.729832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.746595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:09.764574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1047/2088] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:09 2022] POST /identity/v3/projects => generated 357 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:09.768731 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c69272-82c6-41d2-a6a2-837ab0289509 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.779700 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c69272-82c6-41d2-a6a2-837ab0289509 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.839211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.839845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.840144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.840416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.841061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.842995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.857181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:09.884161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1042/2089] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:09 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:53:09.884851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.898178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779c129-4b38-4497-ada7-a4bd83197bc7 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.902961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.903931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.903931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.904126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.904764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.906349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.920628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1048/2090] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:09.921777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.932043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eab718-0e99-4997-8b1c-4ccc90678a9d admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:09.936843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:09.937650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:09.937937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:09.938213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] PATH_INFO: `/v3/projects/c48cb931f63643b9b34ec607dfb4013c/users/3ca6983510494749823b3e4e9781d929/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:09.955162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48cb931f63643b9b34ec607dfb4013c, user_id=3ca6983510494749823b3e4e9781d929, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:09.958049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:09.981848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1043/2091] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:09 2022] PUT /identity/v3/projects/c48cb931f63643b9b34ec607dfb4013c/users/3ca6983510494749823b3e4e9781d929/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:09.985667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:09.998676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132fe27-ccac-44ba-b741-0ec06f7275e1 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:10.068963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008eabc-c5f5-4925-b808-7a78a88be42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:10.069588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008eabc-c5f5-4925-b808-7a78a88be42e admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:10.069868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008eabc-c5f5-4925-b808-7a78a88be42e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:10.070140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008eabc-c5f5-4925-b808-7a78a88be42e admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:10.070790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008eabc-c5f5-4925-b808-7a78a88be42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:10.072414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008eabc-c5f5-4925-b808-7a78a88be42e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:10.082794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1049/2092] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:10.086969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:10.097281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f7c72-bc77-4f8c-8469-9279ebf157aa admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:10.101773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:10.102428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:10.102706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:10.102976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] PATH_INFO: `/v3/projects/c48cb931f63643b9b34ec607dfb4013c/users/3ca6983510494749823b3e4e9781d929/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:10.104912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48cb931f63643b9b34ec607dfb4013c, user_id=3ca6983510494749823b3e4e9781d929, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:10.107616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:10.118219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1044/2093] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:10 2022] PUT /identity/v3/projects/c48cb931f63643b9b34ec607dfb4013c/users/3ca6983510494749823b3e4e9781d929/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:10.128368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:10.128656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:10.128936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:10.173071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] MFA Rules not processed for user `3ca6983510494749823b3e4e9781d929`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:10.174225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:10.250904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1050/2094] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:10.262579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:10.264557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:10.324531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c1851-2f63-4429-813a-36d86abbb912 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:10.328662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:10.329267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:10.329552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:10.329822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:10.339425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:10.340887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:10.351098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1045/2095] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:10.431569 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:10.433328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:10.446560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e9dd5-84c3-4b9d-97fa-06e43297290d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:10.450926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51732732-f4e8-4505-862b-05fbfeb22003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:10.451644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732732-f4e8-4505-862b-05fbfeb22003 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:10.451860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732732-f4e8-4505-862b-05fbfeb22003 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:10.452157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51732732-f4e8-4505-862b-05fbfeb22003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:10.510710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51732732-f4e8-4505-862b-05fbfeb22003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:10.512281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51732732-f4e8-4505-862b-05fbfeb22003 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:10.521521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1051/2096] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:20.346300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:INSERT=3,keystone:SELECT=43 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:20.517250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=98 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:23.659459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:23.661100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:23.729252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cde47-ec2c-413c-954d-814a1946abaf service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:23.734265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:23.734894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:23.735166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:23.735441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] PATH_INFO: `/v3/users/92b64e9d1f15474195f72e079a09efe5` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:23.736619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b64e9d1f15474195f72e079a09efe5)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:23.738889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:23.790314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b64e9d1f15474195f72e079a09efe5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:23.790991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b64e9d1f15474195f72e079a09efe5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:23.798135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b64e9d1f15474195f72e079a09efe5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:23.798785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b64e9d1f15474195f72e079a09efe5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:23.815088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b64e9d1f15474195f72e079a09efe5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:23.830752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1046/2097] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:23 2022] DELETE /identity/v3/users/92b64e9d1f15474195f72e079a09efe5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:23.835612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:23.836148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:23.836241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:23.857165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:23.858375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:23.926202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1052/2098] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:23.939204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:23.940883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:23.999782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7fd2-e978-4b0b-a36a-5afc8ff6053f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:24.003728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:24.004363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:24.004636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:24.004902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:24.014107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:24.015552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:24.024910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1047/2099] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:24.368004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:24.369552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:24.382317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969a51-dacd-49b6-9bae-dbd8b45a0e96 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:24.387379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:24.388120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:24.388393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:24.388758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] PATH_INFO: `/v3/projects/07c537fe35db439c89addc27dc5046eb` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:24.389966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c537fe35db439c89addc27dc5046eb)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:24.392948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:24.440282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c537fe35db439c89addc27dc5046eb', 'request_id': 'req-1fe81581-832e-4606-a2e4-2a496626b1ab'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.448791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c537fe35db439c89addc27dc5046eb', 'request_id': 'req-1fe81581-832e-4606-a2e4-2a496626b1ab'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.455542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c537fe35db439c89addc27dc5046eb', 'request_id': 'req-1fe81581-832e-4606-a2e4-2a496626b1ab'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.474065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] The token cache is being invalidate because project 07c537fe35db439c89addc27dc5046eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:24.474554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.475608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.478174 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1053/2100] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:24 2022] DELETE /identity/v3/projects/07c537fe35db439c89addc27dc5046eb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:24.484237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:24.498391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a5e06-8705-4c59-bd29-ee60bf2d52d7 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:24.517543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:24.557670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:24.558341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:24.558619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:24.558929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] PATH_INFO: `/v3/users/1e7d4989b86c431f8b704ddab3a0c771` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:24.560093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e7d4989b86c431f8b704ddab3a0c771)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:24.562006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:24.621141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e7d4989b86c431f8b704ddab3a0c771'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.621882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e7d4989b86c431f8b704ddab3a0c771'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.630256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e7d4989b86c431f8b704ddab3a0c771'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.630958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e7d4989b86c431f8b704ddab3a0c771'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.643989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-27d3820d-0a57-4538-a110-ceaf98363708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e7d4989b86c431f8b704ddab3a0c771'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:24.672918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1048/2101] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:24 2022] DELETE /identity/v3/users/1e7d4989b86c431f8b704ddab3a0c771 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:24.949568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:24.949568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:24.963814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe81581-832e-4606-a2e4-2a496626b1ab admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.031336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.031991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.032278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.032572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] PATH_INFO: `/v3/projects/f2f427f9dc664b29a61d9a7423acc392` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.033792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f427f9dc664b29a61d9a7423acc392)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.035681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.069136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:25.086282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f427f9dc664b29a61d9a7423acc392', 'request_id': 'req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.092820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f427f9dc664b29a61d9a7423acc392', 'request_id': 'req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.098875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f427f9dc664b29a61d9a7423acc392', 'request_id': 'req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.113483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] The token cache is being invalidate because project f2f427f9dc664b29a61d9a7423acc392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:25.113766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.114381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b83e5d13-9cd7-4425-8b3e-4b1eb416e024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.115909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1054/2102] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:24 2022] DELETE /identity/v3/projects/f2f427f9dc664b29a61d9a7423acc392 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:25.121923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccea73-bafe-4fea-bb8e-f75861bc8d55 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.122210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccea73-bafe-4fea-bb8e-f75861bc8d55 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.122485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccea73-bafe-4fea-bb8e-f75861bc8d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.141769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-3fccea73-bafe-4fea-bb8e-f75861bc8d55 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:25.142965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3fccea73-bafe-4fea-bb8e-f75861bc8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:25.207270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1049/2103] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:25.216042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff938f-9fc1-4132-8f58-43be928b247e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.216328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff938f-9fc1-4132-8f58-43be928b247e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.216572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff938f-9fc1-4132-8f58-43be928b247e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.238464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d5ff938f-9fc1-4132-8f58-43be928b247e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:25.239679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d5ff938f-9fc1-4132-8f58-43be928b247e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:25.248416 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1055/2104] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:25.253134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccea73-bafe-4fea-bb8e-f75861bc8d55 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.263420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccea73-bafe-4fea-bb8e-f75861bc8d55 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.269125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.269724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.270002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.270271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.270921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:25.271221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.272858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.284847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1050/2105] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:25.289349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff938f-9fc1-4132-8f58-43be928b247e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.298728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff938f-9fc1-4132-8f58-43be928b247e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.304644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.305232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.305509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.305771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.306427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.308334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.341760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1056/2106] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:25 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:25.347446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.365875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fa229-1540-446c-9fb3-7108acdccaf0 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.393584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:25.439084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.439814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.440122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.440394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.441032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.442977 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.467863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1051/2107] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:25 2022] POST /identity/v3/users => generated 281 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:25.469446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.480654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d96e5-f907-42eb-9a64-14f7254f6fdf admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.486183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.486852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.487483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.487741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.488426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.490093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.501094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1057/2108] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:25.505129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.515523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2bbab-ca86-43d0-8ece-a5b3ba5f38ba admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.520416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53d8ac-f23f-432d-9a0c-8802d53102f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.521013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d8ac-f23f-432d-9a0c-8802d53102f0 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.521291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d8ac-f23f-432d-9a0c-8802d53102f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.521565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d8ac-f23f-432d-9a0c-8802d53102f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.522211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53d8ac-f23f-432d-9a0c-8802d53102f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.523806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53d8ac-f23f-432d-9a0c-8802d53102f0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.533749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1052/2109] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:25.537869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.547962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87271ac-468a-4dbe-8dca-259a7f9a2d8e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.552637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.553284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.553562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.553835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] PATH_INFO: `/v3/projects/a132eb2b20f34722ad063879a857887f/users/6a91d3f9174440e3815ea445ae1c9713/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.569012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a132eb2b20f34722ad063879a857887f, user_id=6a91d3f9174440e3815ea445ae1c9713, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.571673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.587532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1058/2110] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:25 2022] PUT /identity/v3/projects/a132eb2b20f34722ad063879a857887f/users/6a91d3f9174440e3815ea445ae1c9713/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:25.599114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdee6d4-89e6-4974-ac11-0d33b4aabc24 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.599406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdee6d4-89e6-4974-ac11-0d33b4aabc24 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.599681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdee6d4-89e6-4974-ac11-0d33b4aabc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.628031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9cdee6d4-89e6-4974-ac11-0d33b4aabc24 None None] MFA Rules not processed for user `6a91d3f9174440e3815ea445ae1c9713`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:25.629423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9cdee6d4-89e6-4974-ac11-0d33b4aabc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:25.700237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1053/2111] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:25.734718 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.736556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:25.794827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383fc69-0f3e-4e08-8fb6-8f66a54079d4 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.798717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef5b66-f4ce-471e-a5c1-07d4f3c848f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.799320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef5b66-f4ce-471e-a5c1-07d4f3c848f0 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.799566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef5b66-f4ce-471e-a5c1-07d4f3c848f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.799840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef5b66-f4ce-471e-a5c1-07d4f3c848f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.809266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef5b66-f4ce-471e-a5c1-07d4f3c848f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.810746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef5b66-f4ce-471e-a5c1-07d4f3c848f0 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.821275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1059/2112] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:25.837498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdee6d4-89e6-4974-ac11-0d33b4aabc24 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.900026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdee6d4-89e6-4974-ac11-0d33b4aabc24 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.904785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.905423 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.905722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.905987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] PATH_INFO: `/v3/users/6a91d3f9174440e3815ea445ae1c9713` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.907100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a91d3f9174440e3815ea445ae1c9713)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.909119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.953275 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:25.960098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a91d3f9174440e3815ea445ae1c9713'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.961330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a91d3f9174440e3815ea445ae1c9713'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.971254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a91d3f9174440e3815ea445ae1c9713'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.972022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a91d3f9174440e3815ea445ae1c9713'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:25.984779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a91d3f9174440e3815ea445ae1c9713'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:26.000807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1054/2113] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:25 2022] DELETE /identity/v3/users/6a91d3f9174440e3815ea445ae1c9713 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:26.005423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.005890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.005998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.024835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:26.026022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:26.074832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:26.092296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1060/2114] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:26.102141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:26.103732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:26.163228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d94dc-fca7-41a9-a173-1b7d755920ea admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:26.167355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c368a60-851e-4c75-aaf3-085477f1949a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:26.168037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c368a60-851e-4c75-aaf3-085477f1949a service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.168313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c368a60-851e-4c75-aaf3-085477f1949a service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.168563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c368a60-851e-4c75-aaf3-085477f1949a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.178168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c368a60-851e-4c75-aaf3-085477f1949a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:26.179591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c368a60-851e-4c75-aaf3-085477f1949a service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:26.189644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1055/2115] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:26.473026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:26.479254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:26.496209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0517a-607d-4cc4-8ec5-b4b62fe570ad None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:26.500723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:26.501454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.501805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.502139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] PATH_INFO: `/v3/projects/a132eb2b20f34722ad063879a857887f` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.503414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a132eb2b20f34722ad063879a857887f)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:26.505476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:26.566571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a132eb2b20f34722ad063879a857887f', 'request_id': 'req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:26.573025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a132eb2b20f34722ad063879a857887f', 'request_id': 'req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:26.578964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a132eb2b20f34722ad063879a857887f', 'request_id': 'req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:26.593570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] The token cache is being invalidate because project a132eb2b20f34722ad063879a857887f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:26.593847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:26.594567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f4ff11a4-0cf2-4a0c-8129-8d731d1f7566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:26.596247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1061/2116] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:26 2022] DELETE /identity/v3/projects/a132eb2b20f34722ad063879a857887f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:26.601141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e243c-bf80-4cdf-a987-9cccd656c307 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.601494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e243c-bf80-4cdf-a987-9cccd656c307 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.601765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e243c-bf80-4cdf-a987-9cccd656c307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.621411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9e4e243c-bf80-4cdf-a987-9cccd656c307 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:26.622599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9e4e243c-bf80-4cdf-a987-9cccd656c307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:26.683979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1056/2117] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:26.692843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a5f9c-6b6b-4859-86ed-0d8c28051034 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.693131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a5f9c-6b6b-4859-86ed-0d8c28051034 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.693403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a5f9c-6b6b-4859-86ed-0d8c28051034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.715510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1d1a5f9c-6b6b-4859-86ed-0d8c28051034 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:26.716850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1d1a5f9c-6b6b-4859-86ed-0d8c28051034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:26.725250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1062/2118] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:26.730556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e243c-bf80-4cdf-a987-9cccd656c307 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:26.749512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e243c-bf80-4cdf-a987-9cccd656c307 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:26.754579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:26.755141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.755418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.755686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.756404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:26.756667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:26.758246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:26.770463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1057/2119] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:26.774768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a5f9c-6b6b-4859-86ed-0d8c28051034 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:26.785535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a5f9c-6b6b-4859-86ed-0d8c28051034 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:26.791231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:26.791840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.792262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.792542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.793191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:26.795080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:26.827272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1063/2120] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:26 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:53:26.831268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:26.841844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259e27-c6cc-4cf2-8391-b4dbd5a523cb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:26.887852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:26.905429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:26.906049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.906324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.906588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.907265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:26.909186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:26.939497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1058/2121] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:26 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:26.940763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:26.962408 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8137-5fb0-489d-a21f-80f57458b612 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:26.972608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:26.973237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:26.973521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:26.973788 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:26.974446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:26.976068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:26.994006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1064/2122] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:26.995212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:27.006851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c9e17-8130-4c07-8ad5-64d3743fd32a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:27.012780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16311e05-8a6b-4a1e-924c-aa214cf7bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:27.013166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16311e05-8a6b-4a1e-924c-aa214cf7bc8c admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:27.013466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16311e05-8a6b-4a1e-924c-aa214cf7bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:27.013771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16311e05-8a6b-4a1e-924c-aa214cf7bc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:27.014480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16311e05-8a6b-4a1e-924c-aa214cf7bc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:27.017081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16311e05-8a6b-4a1e-924c-aa214cf7bc8c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:27.028235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1059/2123] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:53:27.036373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:27.050312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff921f-ca8e-4854-841a-18c1c79f6b07 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:27.056272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:27.057064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:27.057575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:27.058046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] PATH_INFO: `/v3/projects/0f41b1696e934fd78f57519c7994e5b3/users/4a031835644f46038310afeba5722b78/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:27.076000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f41b1696e934fd78f57519c7994e5b3, user_id=4a031835644f46038310afeba5722b78, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:27.079167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:27.095959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1065/2124] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:27 2022] PUT /identity/v3/projects/0f41b1696e934fd78f57519c7994e5b3/users/4a031835644f46038310afeba5722b78/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:27.172088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:27.172345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:27.172623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:27.207915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] MFA Rules not processed for user `4a031835644f46038310afeba5722b78`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:27.209166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:27.272192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1060/2125] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:27.284953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:27.286707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:27.349770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d457ee5-b6cc-4438-bff1-417182a62f42 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:27.354306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb2d33-d028-4704-bdb7-4efd373a393b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:27.354911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb2d33-d028-4704-bdb7-4efd373a393b service glance] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:27.355189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb2d33-d028-4704-bdb7-4efd373a393b service glance] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:27.355456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb2d33-d028-4704-bdb7-4efd373a393b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:27.364883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb2d33-d028-4704-bdb7-4efd373a393b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:27.366308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb2d33-d028-4704-bdb7-4efd373a393b service glance] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:27.375594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1066/2126] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:27.537985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:27.539457 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:27.619530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a95e67-8447-4055-9b3a-63decfb1fbb0 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:27.624400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:27.625162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:27.625507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:27.625853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] PATH_INFO: `/v3/users/d81f4a50a67d4dd7bf8e590c924db1ef` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:27.627140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81f4a50a67d4dd7bf8e590c924db1ef)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:27.629198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:27.689458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81f4a50a67d4dd7bf8e590c924db1ef'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:27.690144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81f4a50a67d4dd7bf8e590c924db1ef'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:27.697429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81f4a50a67d4dd7bf8e590c924db1ef'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:27.698500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81f4a50a67d4dd7bf8e590c924db1ef'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:27.700574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:27.722969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81f4a50a67d4dd7bf8e590c924db1ef'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:27.746440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1061/2127] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:27 2022] DELETE /identity/v3/users/d81f4a50a67d4dd7bf8e590c924db1ef => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:27.750811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:27.751084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:27.751357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:27.804390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:27.806543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:27.852737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:27.969010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1067/2128] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:27.983636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:27.985252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:28.061450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694a749-a40b-44b7-8c46-222a1910f547 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:28.065232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:28.065846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:28.066116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:28.066393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:28.077261 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:28.078850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:28.089025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1062/2129] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:28.632318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:28.634696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:28.660527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e321b37-7e6f-44fd-89b7-2cd8a968bae3 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:28.688638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:28.689441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:28.690332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:28.690602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] PATH_INFO: `/v3/projects/6aae9acee72c40df93a948a7dfe006ae` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:28.692477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aae9acee72c40df93a948a7dfe006ae)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:28.696013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:28.844355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aae9acee72c40df93a948a7dfe006ae', 'request_id': 'req-c2c9d92a-fca1-48dd-8680-72d14b9e2528'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:28.851835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aae9acee72c40df93a948a7dfe006ae', 'request_id': 'req-c2c9d92a-fca1-48dd-8680-72d14b9e2528'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:28.858552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aae9acee72c40df93a948a7dfe006ae', 'request_id': 'req-c2c9d92a-fca1-48dd-8680-72d14b9e2528'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:28.882667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] The token cache is being invalidate because project 6aae9acee72c40df93a948a7dfe006ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:28.883145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:28.884224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:28.887368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1068/2130] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:28 2022] DELETE /identity/v3/projects/6aae9acee72c40df93a948a7dfe006ae => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:28.891701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:28.903316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd79b6-9023-41b3-913a-c93ed10acc4c service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:28.981574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:28.982247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:28.982521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:28.982807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] PATH_INFO: `/v3/users/de41cfe4c4fd4f488c178ae4131a62b1` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:28.983946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de41cfe4c4fd4f488c178ae4131a62b1)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:28.985881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.035912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de41cfe4c4fd4f488c178ae4131a62b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.036580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de41cfe4c4fd4f488c178ae4131a62b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.045323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de41cfe4c4fd4f488c178ae4131a62b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.045972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de41cfe4c4fd4f488c178ae4131a62b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.058421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-674bf501-d6e0-422d-9c59-24e76cbec878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de41cfe4c4fd4f488c178ae4131a62b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.062052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:29.075112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1063/2131] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:28 2022] DELETE /identity/v3/users/de41cfe4c4fd4f488c178ae4131a62b1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:29.350017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.359707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:29.375806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9d92a-fca1-48dd-8680-72d14b9e2528 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.434893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.435437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.435715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.436012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] PATH_INFO: `/v3/projects/47db851d76ca40f79a30bbc5d5d44e0f` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.437129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=47db851d76ca40f79a30bbc5d5d44e0f)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.439048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.499098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:29.507592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47db851d76ca40f79a30bbc5d5d44e0f', 'request_id': 'req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.516891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47db851d76ca40f79a30bbc5d5d44e0f', 'request_id': 'req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.524827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47db851d76ca40f79a30bbc5d5d44e0f', 'request_id': 'req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.539160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] The token cache is being invalidate because project 47db851d76ca40f79a30bbc5d5d44e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:29.539450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.540125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d2e9c6c-1196-45a8-90ac-cc41e5eae09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.542495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1069/2132] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:29 2022] DELETE /identity/v3/projects/47db851d76ca40f79a30bbc5d5d44e0f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:29.547410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3656309-db55-477d-b1aa-6350984ea91c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.547813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3656309-db55-477d-b1aa-6350984ea91c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.548184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3656309-db55-477d-b1aa-6350984ea91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.576417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c3656309-db55-477d-b1aa-6350984ea91c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:29.577634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c3656309-db55-477d-b1aa-6350984ea91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:29.638549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1064/2133] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:29.647082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6cbbf-3c34-4da9-bb70-17033572811b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.647618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6cbbf-3c34-4da9-bb70-17033572811b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.647697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6cbbf-3c34-4da9-bb70-17033572811b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.670991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-03e6cbbf-3c34-4da9-bb70-17033572811b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:29.672249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-03e6cbbf-3c34-4da9-bb70-17033572811b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:29.682035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1070/2134] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:29.686905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3656309-db55-477d-b1aa-6350984ea91c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.696831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3656309-db55-477d-b1aa-6350984ea91c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.701576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.702182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.702460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.702732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.703387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:29.703821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.707414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.719276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1065/2135] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:29.723831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6cbbf-3c34-4da9-bb70-17033572811b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.733733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6cbbf-3c34-4da9-bb70-17033572811b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.738871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.739473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.739755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.740067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.740715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.742850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.779171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1071/2136] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:29 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:29.788985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.800270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4018f4-b6a4-46f7-a18f-5f13e273b054 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.861035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.861649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.861930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.862543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.863178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.865140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.889676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1066/2137] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:29 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:29.895686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.909930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ddfe0-857f-4fb2-96c7-52fc4060603f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.914922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.915533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.915808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.916167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.916883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.918581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.928887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1072/2138] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:29.932955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.943279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659abde-219e-47f3-8db6-2714928b796a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.948332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38d094-cf58-4f9b-bbb0-20f4cca9514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.948939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38d094-cf58-4f9b-bbb0-20f4cca9514a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.949217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38d094-cf58-4f9b-bbb0-20f4cca9514a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.949662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38d094-cf58-4f9b-bbb0-20f4cca9514a admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.950310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38d094-cf58-4f9b-bbb0-20f4cca9514a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.952094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38d094-cf58-4f9b-bbb0-20f4cca9514a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.963749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1067/2139] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:29.980879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.993867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c77af8-3fd7-4f06-9577-69389d438978 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.998651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.999312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.999593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.999867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] PATH_INFO: `/v3/projects/d3918157572341b29b13966b7dd29c9b/users/f073cf9e464b456fb1f0ba3fd6e8a666/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.019261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3918157572341b29b13966b7dd29c9b, user_id=f073cf9e464b456fb1f0ba3fd6e8a666, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.022716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.064792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1073/2140] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:29 2022] PUT /identity/v3/projects/d3918157572341b29b13966b7dd29c9b/users/f073cf9e464b456fb1f0ba3fd6e8a666/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:30.068721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e06c7-3b47-4785-99c9-1eca53c95c1c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.068971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e06c7-3b47-4785-99c9-1eca53c95c1c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.069460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e06c7-3b47-4785-99c9-1eca53c95c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.101834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0d6e06c7-3b47-4785-99c9-1eca53c95c1c None None] MFA Rules not processed for user `f073cf9e464b456fb1f0ba3fd6e8a666`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:30.103211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0d6e06c7-3b47-4785-99c9-1eca53c95c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:30.166802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:30.187193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1068/2141] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:30.191674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.267027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe041c-fb82-4255-bef9-e72abe568488 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.278431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.279073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.279352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.279638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.280483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.282765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.318019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1074/2142] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:30 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:53:30.319807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e06c7-3b47-4785-99c9-1eca53c95c1c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.330554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e06c7-3b47-4785-99c9-1eca53c95c1c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.391316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.392056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.392339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.392610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.393249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.395171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.418394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1069/2143] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:30 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:30.421741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.433936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad8238-32c3-42f9-9aa2-45b18b108b46 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.443011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.443686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.444063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.444321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.444979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.446749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.459416 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1075/2144] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:30.463913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.474734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d1ca0-c9a6-4693-bc89-489346be69ab admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.480308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.480953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.481247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.481509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] PATH_INFO: `/v3/projects/f6cf6268d2184b7891762f6f1c0121fc/users/a6b50b232f58489093ee8b40e91f0068/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.497999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6cf6268d2184b7891762f6f1c0121fc, user_id=a6b50b232f58489093ee8b40e91f0068, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.500866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.519740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1070/2145] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:30 2022] PUT /identity/v3/projects/f6cf6268d2184b7891762f6f1c0121fc/users/a6b50b232f58489093ee8b40e91f0068/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:30.523807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.534902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893036f4-a93f-4426-b59d-5b2eb3ae4d52 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.579491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:30.599826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bc646-7678-49c3-b5e9-134a2304740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.600449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bc646-7678-49c3-b5e9-134a2304740f admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.600729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bc646-7678-49c3-b5e9-134a2304740f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.600999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bc646-7678-49c3-b5e9-134a2304740f admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.601639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bc646-7678-49c3-b5e9-134a2304740f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.603351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bc646-7678-49c3-b5e9-134a2304740f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.614243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1076/2146] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:30.618673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.630952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6bf48-4f1d-420b-9ac1-f92053cffb0a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.635846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.636614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.636955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.638678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] PATH_INFO: `/v3/projects/f6cf6268d2184b7891762f6f1c0121fc/users/a6b50b232f58489093ee8b40e91f0068/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.639341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6cf6268d2184b7891762f6f1c0121fc, user_id=a6b50b232f58489093ee8b40e91f0068, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.642276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.655153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1071/2147] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:30 2022] PUT /identity/v3/projects/f6cf6268d2184b7891762f6f1c0121fc/users/a6b50b232f58489093ee8b40e91f0068/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:30.666263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.666810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.666893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.696072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] MFA Rules not processed for user `a6b50b232f58489093ee8b40e91f0068`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:30.697294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:30.772856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1077/2148] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:30.787028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.793055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:30.852936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9f3f2-6a67-4eb6-84ff-7766310ac1b3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.857544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.858205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.858484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.858752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:30.877888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:30.879381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:30.890137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1072/2149] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:30.973047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:30.974469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:30.993564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43145d7-8d5d-431d-982b-77363f2665b1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:30.997202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b91a7-01d5-45ea-9653-b820e2602f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:30.997813 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b91a7-01d5-45ea-9653-b820e2602f05 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:30.998094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b91a7-01d5-45ea-9653-b820e2602f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:30.998425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b91a7-01d5-45ea-9653-b820e2602f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:31.061771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b91a7-01d5-45ea-9653-b820e2602f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:31.063452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b91a7-01d5-45ea-9653-b820e2602f05 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:31.074953 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1078/2150] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:34.744170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:34.800539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:34.822199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7f262-1909-43f9-bf96-e4866787b7fc service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:34.827612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:34.828475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:34.828855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:34.829289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] PATH_INFO: `/v3/users/f073cf9e464b456fb1f0ba3fd6e8a666` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:34.830708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f073cf9e464b456fb1f0ba3fd6e8a666)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:34.833106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:34.898751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f073cf9e464b456fb1f0ba3fd6e8a666'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:34.899414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f073cf9e464b456fb1f0ba3fd6e8a666'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:34.906348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f073cf9e464b456fb1f0ba3fd6e8a666'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:34.907000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f073cf9e464b456fb1f0ba3fd6e8a666'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:34.919847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f073cf9e464b456fb1f0ba3fd6e8a666'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:34.935348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1073/2151] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:34 2022] DELETE /identity/v3/users/f073cf9e464b456fb1f0ba3fd6e8a666 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:34.941296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:34.942492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:34.942770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:34.969265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:34.971147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:35.035393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:35.046533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1079/2152] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:35.058976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:35.066171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:35.129787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaf201-6320-44ce-9dd7-ddd467dc6cd0 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:35.134047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:35.134834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:35.135218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:35.136120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:35.147398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:35.148985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:35.159545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1074/2153] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:35.667396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:35.670002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:35.699736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a1b76-a9b1-4464-a137-1c112cc31740 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:35.704787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:35.705550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:35.705684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:35.705969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] PATH_INFO: `/v3/projects/d3918157572341b29b13966b7dd29c9b` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:35.707069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3918157572341b29b13966b7dd29c9b)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:35.709164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:35.803627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3918157572341b29b13966b7dd29c9b', 'request_id': 'req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.816753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3918157572341b29b13966b7dd29c9b', 'request_id': 'req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.826002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3918157572341b29b13966b7dd29c9b', 'request_id': 'req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.851629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] The token cache is being invalidate because project d3918157572341b29b13966b7dd29c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:35.852089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.853159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-9b14f7d8-fbd7-44d5-8e9a-3a1145e58b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.855202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1080/2154] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:35 2022] DELETE /identity/v3/projects/d3918157572341b29b13966b7dd29c9b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:35.859154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:35.872286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f8e70-2ddb-4284-953b-2587ecf7a88d service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:35.941155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:35.941803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:35.942077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:35.942934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] PATH_INFO: `/v3/users/a6b50b232f58489093ee8b40e91f0068` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:35.944152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b50b232f58489093ee8b40e91f0068)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:35.950068 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:35.960721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:36.017638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b50b232f58489093ee8b40e91f0068'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.018596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b50b232f58489093ee8b40e91f0068'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.026235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b50b232f58489093ee8b40e91f0068'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.026960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b50b232f58489093ee8b40e91f0068'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.039051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-22258bc0-e4d9-4bb0-a63f-9e99f4cd6eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b50b232f58489093ee8b40e91f0068'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.053155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1075/2155] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:35 2022] DELETE /identity/v3/users/a6b50b232f58489093ee8b40e91f0068 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:36.211299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34b62-b191-41b2-a541-447ce0b74b5c None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.211843 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34b62-b191-41b2-a541-447ce0b74b5c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.212062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34b62-b191-41b2-a541-447ce0b74b5c None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.213283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1081/2156] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:53:36.251200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.251823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.252208 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.273945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.275784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.293123 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34b62-b191-41b2-a541-447ce0b74b5c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.295146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-38d34b62-b191-41b2-a541-447ce0b74b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.371344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1076/2157] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:36.372351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.373771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.385256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34b62-b191-41b2-a541-447ce0b74b5c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.392009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156af88a-82e2-49cd-a994-fdd613b5b9d7 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.393697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.394474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.394866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.395107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] PATH_INFO: `/v3/users/91ccd41938954a36adb45c70e6ff41de` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.396308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ccd41938954a36adb45c70e6ff41de)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.397379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.398009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.398313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.398471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.398562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] PATH_INFO: `/v3/projects/f6cf6268d2184b7891762f6f1c0121fc` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.399759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6cf6268d2184b7891762f6f1c0121fc)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.402357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.453529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ccd41938954a36adb45c70e6ff41de'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.454436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ccd41938954a36adb45c70e6ff41de'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.455131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ccd41938954a36adb45c70e6ff41de'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.456801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6cf6268d2184b7891762f6f1c0121fc', 'request_id': 'req-09527863-344a-405a-9456-c62819c8bce6'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.463269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6cf6268d2184b7891762f6f1c0121fc', 'request_id': 'req-09527863-344a-405a-9456-c62819c8bce6'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.463690 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ccd41938954a36adb45c70e6ff41de'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.469964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6cf6268d2184b7891762f6f1c0121fc', 'request_id': 'req-09527863-344a-405a-9456-c62819c8bce6'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.477744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af91114b-0001-43ba-b31b-b3a4bfd41dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ccd41938954a36adb45c70e6ff41de'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.494986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] The token cache is being invalidate because project f6cf6268d2184b7891762f6f1c0121fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:36.495265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.496062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-09527863-344a-405a-9456-c62819c8bce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.497604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1077/2158] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:36 2022] DELETE /identity/v3/projects/f6cf6268d2184b7891762f6f1c0121fc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:36.499438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686018e-0e5e-411c-92f4-65254d40dd99 None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.499720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686018e-0e5e-411c-92f4-65254d40dd99 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.500024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686018e-0e5e-411c-92f4-65254d40dd99 None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.501813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1078/2159] 10.222.0.44 () {54 vars in 961 bytes} [Tue Oct 4 08:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:53:36.503240 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7003-8a0a-40b8-82e3-a112ba6487d8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.503521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7003-8a0a-40b8-82e3-a112ba6487d8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.503848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7003-8a0a-40b8-82e3-a112ba6487d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.509514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1082/2160] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:36 2022] DELETE /identity/v3/users/91ccd41938954a36adb45c70e6ff41de => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:36.510983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a2aa3-f194-4813-babc-7cf86c37ce3a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.511269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a2aa3-f194-4813-babc-7cf86c37ce3a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.511539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a2aa3-f194-4813-babc-7cf86c37ce3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.519610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:36.524713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:36.532021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-100a7003-8a0a-40b8-82e3-a112ba6487d8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.533433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9d3a2aa3-f194-4813-babc-7cf86c37ce3a None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.535539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-100a7003-8a0a-40b8-82e3-a112ba6487d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.535713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9d3a2aa3-f194-4813-babc-7cf86c37ce3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.614320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1079/2161] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:36.622073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ceb5-6e6b-4fbe-9dda-535cc68fc64d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.622451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ceb5-6e6b-4fbe-9dda-535cc68fc64d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.622727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ceb5-6e6b-4fbe-9dda-535cc68fc64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.644162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1083/2162] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:36.645266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a2aa3-f194-4813-babc-7cf86c37ce3a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.647216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4753ceb5-6e6b-4fbe-9dda-535cc68fc64d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.648464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4753ceb5-6e6b-4fbe-9dda-535cc68fc64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.658183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1080/2163] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:36.659467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a2aa3-f194-4813-babc-7cf86c37ce3a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.662368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ceb5-6e6b-4fbe-9dda-535cc68fc64d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.673871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ceb5-6e6b-4fbe-9dda-535cc68fc64d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.676656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.678085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.678374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.683974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.684218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.684218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.684218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.684218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.684617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.684693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:36.684693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.684693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.697004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1081/2164] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:36.698076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.699698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.701281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.713307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1084/2165] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:36.717443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11cb9b-5fb2-4ca5-9362-38b3e7db0eb6 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.724879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.725722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.726004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.726320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.727108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.729072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.746051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.747575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.777885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b234d3-bf36-4095-94ac-cd7b9d69b76d service placement] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.784063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1082/2166] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/projects => generated 401 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:36.785510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.851132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab9f98-1ebc-46a7-8421-1662a41ba460 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.856033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.856807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.857226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.857674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.858531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.860678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.861809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.863100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.863386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.863663 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] PATH_INFO: `/v3/projects/0c4d8cf4a0b74162b6f73a0b2d88ff09` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.864899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4d8cf4a0b74162b6f73a0b2d88ff09)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.866798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.887088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1083/2167] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:53:36.891230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.901795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1b88e-0c6e-4109-8413-adea472a2771 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.906543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.907192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.907528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.907768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.908453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.910090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.920915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1084/2168] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:36.924882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.928267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4d8cf4a0b74162b6f73a0b2d88ff09', 'request_id': 'req-f3f17012-a632-4f1d-b7a6-1704fb314c89'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.934375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610da6e-be4c-4748-846e-13b08eeda867 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.935418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4d8cf4a0b74162b6f73a0b2d88ff09', 'request_id': 'req-f3f17012-a632-4f1d-b7a6-1704fb314c89'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.941025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4d8cf4a0b74162b6f73a0b2d88ff09', 'request_id': 'req-f3f17012-a632-4f1d-b7a6-1704fb314c89'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.956148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] The token cache is being invalidate because project 0c4d8cf4a0b74162b6f73a0b2d88ff09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:36.956589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.957260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3f17012-a632-4f1d-b7a6-1704fb314c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.958813 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1085/2169] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:36 2022] DELETE /identity/v3/projects/0c4d8cf4a0b74162b6f73a0b2d88ff09 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:36.964445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a0be0-3b70-4977-8906-d577ef111c22 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.964724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a0be0-3b70-4977-8906-d577ef111c22 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.964990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a0be0-3b70-4977-8906-d577ef111c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.978835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:36.986014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-960a0be0-3b70-4977-8906-d577ef111c22 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.988989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-960a0be0-3b70-4977-8906-d577ef111c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.996615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.997227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.997797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.998081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.998730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.000359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:37.006153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.023286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1085/2170] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:53:37.027669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.030484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.044888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b1aee-ea78-421d-8266-e27d4c55cd5f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.049994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a47b1d-661f-413b-af3e-86324eb1a873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.050780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47b1d-661f-413b-af3e-86324eb1a873 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.051119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47b1d-661f-413b-af3e-86324eb1a873 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.051703 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47b1d-661f-413b-af3e-86324eb1a873 admin admin] PATH_INFO: `/v3/projects/162de8ca3d3e40da9233bdb44aaf8ebe/users/20ee1a29f5ef4d52adf45f926406b367/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.065344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1086/2171] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:37.069722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a47b1d-661f-413b-af3e-86324eb1a873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162de8ca3d3e40da9233bdb44aaf8ebe, user_id=20ee1a29f5ef4d52adf45f926406b367, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.072542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a47b1d-661f-413b-af3e-86324eb1a873 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.073056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dd8ec-10eb-491c-94cd-3b3064a38cc6 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.073334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dd8ec-10eb-491c-94cd-3b3064a38cc6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.073592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dd8ec-10eb-491c-94cd-3b3064a38cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.091562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1086/2172] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:37 2022] PUT /identity/v3/projects/162de8ca3d3e40da9233bdb44aaf8ebe/users/20ee1a29f5ef4d52adf45f926406b367/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:37.094498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c64dd8ec-10eb-491c-94cd-3b3064a38cc6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:37.097273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c64dd8ec-10eb-491c-94cd-3b3064a38cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.103545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556749-df98-4414-ae6b-9fbb7ca96e02 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.103545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556749-df98-4414-ae6b-9fbb7ca96e02 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.103545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556749-df98-4414-ae6b-9fbb7ca96e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.131570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ae556749-df98-4414-ae6b-9fbb7ca96e02 None None] MFA Rules not processed for user `20ee1a29f5ef4d52adf45f926406b367`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:37.133088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ae556749-df98-4414-ae6b-9fbb7ca96e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.180382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1087/2173] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:37.184784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dd8ec-10eb-491c-94cd-3b3064a38cc6 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.201584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dd8ec-10eb-491c-94cd-3b3064a38cc6 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.211118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.211778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.212382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.212493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.213220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:37.213525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.216063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.231936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1087/2174] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:37.233612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1088/2175] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:37.234660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556749-df98-4414-ae6b-9fbb7ca96e02 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.235104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.247426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21cf0d-cec6-4c1f-a3b8-5bfb4fc83848 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.251733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556749-df98-4414-ae6b-9fbb7ca96e02 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.253030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.257283 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.258106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.258501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.258861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.259602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.260669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.261291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.261927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.262184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.263060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.265715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.299954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1088/2176] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:37.304123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.304456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1089/2177] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/projects => generated 399 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:37.308544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.384891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505c4a1-97d1-40b9-8f71-2dca78f63a31 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.387439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e52b88-1aa2-485c-ba69-bded82ab6694 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.391027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.391834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.392159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.392431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.393176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.395144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.396468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.397125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.397400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.397673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.398362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.400342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.424540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1090/2178] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/users => generated 306 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:53:37.424540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.424818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1089/2179] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:37.426153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.427257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.438455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde483b9-2857-4622-9957-7ed44bbb793a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.439475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed094f-ff5b-4c70-bfab-c2671d062ca5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.443273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.443872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.444178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.444443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.444862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.445084 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.445491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.445768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.446040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] PATH_INFO: `/v3/users/68a43aad622e429eb504e4c3307a116b` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.446655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.447160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a43aad622e429eb504e4c3307a116b)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.449145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.455811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:37.458830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1090/2180] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:37.459703 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.465479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:37.472014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7269-704e-4181-a4c6-f81a0b609cfd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.482098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.482738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.483014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.483376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.484069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.485711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.503455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1091/2181] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:37.503455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.513353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a43aad622e429eb504e4c3307a116b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:37.514251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a43aad622e429eb504e4c3307a116b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:37.514932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb441e0-2691-4938-be72-0fa1248d174f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.515251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a43aad622e429eb504e4c3307a116b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:37.524420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af1aaa-142a-4741-87fd-a6c2b43ed5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.524632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a43aad622e429eb504e4c3307a116b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:37.525221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1aaa-142a-4741-87fd-a6c2b43ed5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.525558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1aaa-142a-4741-87fd-a6c2b43ed5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.525978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1aaa-142a-4741-87fd-a6c2b43ed5ba admin admin] PATH_INFO: `/v3/projects/07109830f0a8421186d7c6976e9a9b64/users/2bb63d3526a34964a666b317f2523817/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.538160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a43aad622e429eb504e4c3307a116b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:37.543123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af1aaa-142a-4741-87fd-a6c2b43ed5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=07109830f0a8421186d7c6976e9a9b64, user_id=2bb63d3526a34964a666b317f2523817, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.556266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af1aaa-142a-4741-87fd-a6c2b43ed5ba admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.556384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1091/2182] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:37 2022] DELETE /identity/v3/users/68a43aad622e429eb504e4c3307a116b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:37.556384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.560173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.563455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1092/2183] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:37 2022] PUT /identity/v3/projects/07109830f0a8421186d7c6976e9a9b64/users/2bb63d3526a34964a666b317f2523817/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:37.564687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.564952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.565221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.589310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:37.590550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.625526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0cd3-74d1-4a0c-a195-34983af511f6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.648159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1093/2184] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:37.648934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.657203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.669206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85129c4d-34f4-41d3-b06a-d95910260182 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.674281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.674887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.675158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.675425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.676226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.678540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.690269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1094/2185] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:37.691131 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.692776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.695186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.695862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.696215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.696491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.697149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.698765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.709202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1092/2186] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:37.710051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.719776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed08fb1-1c81-4ca7-9215-5f323b0c6feb admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.724502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.725139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.725410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.725677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] PATH_INFO: `/v3/projects/07109830f0a8421186d7c6976e9a9b64/users/2bb63d3526a34964a666b317f2523817/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.727541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07109830f0a8421186d7c6976e9a9b64, user_id=2bb63d3526a34964a666b317f2523817, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.730227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.740934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1093/2187] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:37 2022] PUT /identity/v3/projects/07109830f0a8421186d7c6976e9a9b64/users/2bb63d3526a34964a666b317f2523817/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:37.741748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.758944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433b391-7ccb-4e69-9dbd-cf407c0606cf admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.767171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:37.813723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54547ba-8ae6-42fe-9010-fd7a3c5b7144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.814440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54547ba-8ae6-42fe-9010-fd7a3c5b7144 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.814719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54547ba-8ae6-42fe-9010-fd7a3c5b7144 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.814987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54547ba-8ae6-42fe-9010-fd7a3c5b7144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.820172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223283a4-313d-4f51-b9ab-026945eff9f6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.825021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5966fe8-a3bf-4ae8-b805-a5313370a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.825248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54547ba-8ae6-42fe-9010-fd7a3c5b7144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.825711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966fe8-a3bf-4ae8-b805-a5313370a131 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.825986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966fe8-a3bf-4ae8-b805-a5313370a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.826481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966fe8-a3bf-4ae8-b805-a5313370a131 admin admin] PATH_INFO: `/v3/projects/b7ad37ea522247c6b28ea40e1bc43669/users/e71984f02fcd4deba5e1430ec85b8705/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.826708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54547ba-8ae6-42fe-9010-fd7a3c5b7144 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.836547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1095/2188] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:37.837789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.838074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.838345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.842101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5966fe8-a3bf-4ae8-b805-a5313370a131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ad37ea522247c6b28ea40e1bc43669, user_id=e71984f02fcd4deba5e1430ec85b8705, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.845011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5966fe8-a3bf-4ae8-b805-a5313370a131 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.863863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1094/2189] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:37 2022] PUT /identity/v3/projects/b7ad37ea522247c6b28ea40e1bc43669/users/e71984f02fcd4deba5e1430ec85b8705/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:37.869347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.869670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.870050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.879973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] MFA Rules not processed for user `2bb63d3526a34964a666b317f2523817`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:37.881207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.885514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:37.892116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:37.893580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.983910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1096/2190] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:37.995804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:37.996238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1095/2191] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:37.997614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:38.006579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:38.009265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:38.078619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4c21-42c6-414b-a199-aae4d132d09b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:38.084766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ce1cd-3f47-4875-9514-c89a8e85b2f8 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:38.089659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:38.100241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:38.100241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:38.100241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:38.100542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:38.100542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:38.100542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:38.100542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:38.112865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:38.114632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:38.121903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:38.123422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:38.126423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1096/2192] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:53:38.135408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1097/2193] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:37 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:38.221899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:38.224152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:38.238004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f58570-2381-44cf-95f7-45b9c8e04814 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:38.243191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:38.243927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:38.244788 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:38.245391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:38.247420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:38.293260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:38.316820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:38.318318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:38.323416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:38.325302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:38.330412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1097/2194] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:38.339798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785288-2dac-455b-a3dc-2e0b86a3b48a service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:38.352826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:38.353646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:38.353928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:38.354195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] PATH_INFO: `/v3/projects/739295c6103a427094026324bdc86f37` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:38.355738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=739295c6103a427094026324bdc86f37)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:38.357812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:38.429697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '739295c6103a427094026324bdc86f37', 'request_id': 'req-2e63c385-43a8-4e29-bea0-58ab849b40ff'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.442214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '739295c6103a427094026324bdc86f37', 'request_id': 'req-2e63c385-43a8-4e29-bea0-58ab849b40ff'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.449708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '739295c6103a427094026324bdc86f37', 'request_id': 'req-2e63c385-43a8-4e29-bea0-58ab849b40ff'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.466982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] The token cache is being invalidate because project 739295c6103a427094026324bdc86f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:38.467432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.468377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.470213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1098/2195] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:38 2022] DELETE /identity/v3/projects/739295c6103a427094026324bdc86f37 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:38.474163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:38.487623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92971-74c9-4cab-9c04-7737c9c3a639 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:38.649340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:38.651923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:38.652272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:38.653125 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] PATH_INFO: `/v3/users/3ca6983510494749823b3e4e9781d929` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:38.658393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca6983510494749823b3e4e9781d929)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:38.663191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:38.761141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca6983510494749823b3e4e9781d929'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.761899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca6983510494749823b3e4e9781d929'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.762486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca6983510494749823b3e4e9781d929'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.771367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca6983510494749823b3e4e9781d929'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.788879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-27b6144d-9619-49cf-b256-c33bb6cb17f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca6983510494749823b3e4e9781d929'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:38.806279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1098/2196] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:38 2022] DELETE /identity/v3/users/3ca6983510494749823b3e4e9781d929 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:39.678436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:39.679900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:39.700308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c385-43a8-4e29-bea0-58ab849b40ff admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:39.743556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:39.844936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:39.845733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:39.845991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:39.846260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] PATH_INFO: `/v3/projects/c48cb931f63643b9b34ec607dfb4013c` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:39.847444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48cb931f63643b9b34ec607dfb4013c)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:39.850166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:39.947720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48cb931f63643b9b34ec607dfb4013c', 'request_id': 'req-1bc07458-453d-4601-9bec-78d53f86497d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:39.957282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48cb931f63643b9b34ec607dfb4013c', 'request_id': 'req-1bc07458-453d-4601-9bec-78d53f86497d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:39.963955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48cb931f63643b9b34ec607dfb4013c', 'request_id': 'req-1bc07458-453d-4601-9bec-78d53f86497d'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.000673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] The token cache is being invalidate because project c48cb931f63643b9b34ec607dfb4013c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:40.001151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.002666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1bc07458-453d-4601-9bec-78d53f86497d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.004821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1099/2197] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:39 2022] DELETE /identity/v3/projects/c48cb931f63643b9b34ec607dfb4013c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:40.010256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70b022-0587-49b8-9d3d-f528ff656ae2 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.010763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70b022-0587-49b8-9d3d-f528ff656ae2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.010842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70b022-0587-49b8-9d3d-f528ff656ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.044451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-df70b022-0587-49b8-9d3d-f528ff656ae2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:40.045681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-df70b022-0587-49b8-9d3d-f528ff656ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.147826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:40.170444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1099/2198] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:40.179351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b9bb-2760-40ac-8199-3ddc2c8128a0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.180006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b9bb-2760-40ac-8199-3ddc2c8128a0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.180236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b9bb-2760-40ac-8199-3ddc2c8128a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.215574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-cee1b9bb-2760-40ac-8199-3ddc2c8128a0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:40.217145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-cee1b9bb-2760-40ac-8199-3ddc2c8128a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.239005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1100/2199] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:40.239995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70b022-0587-49b8-9d3d-f528ff656ae2 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.264868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70b022-0587-49b8-9d3d-f528ff656ae2 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.278007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.278640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.278915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.279187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.279919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:40.280248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.282285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.295403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1100/2200] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:40.300202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1b9bb-2760-40ac-8199-3ddc2c8128a0 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.313497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1b9bb-2760-40ac-8199-3ddc2c8128a0 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.322812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.323442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.323716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.324008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.324645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.329873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.379967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1101/2201] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:40.383957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.396231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ba582-8736-45ee-97a9-0de49ec223f3 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.486260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.487702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.488242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.489258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.490629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.493373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.523926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1101/2202] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:40.526116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.536722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe29cd2-df15-45e8-ab0d-533120ab79d3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.541947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.542562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.542843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.543115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.543792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.545679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.557157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1102/2203] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:40.561261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.571617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19630-e1fc-4b12-84a6-30b12e8f812f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.576315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0632-5138-4b8d-855e-cf365f3b4f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.576921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0632-5138-4b8d-855e-cf365f3b4f34 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.577199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0632-5138-4b8d-855e-cf365f3b4f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.577467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0632-5138-4b8d-855e-cf365f3b4f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.578113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0632-5138-4b8d-855e-cf365f3b4f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.579832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0632-5138-4b8d-855e-cf365f3b4f34 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.592299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1102/2204] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:53:40.596528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.608796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d1f7-d28a-4ae7-a6e4-0a18d077b9c8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.614986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.615748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.616119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.616350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] PATH_INFO: `/v3/projects/c4736c37dc824497ad9213da3a83d633/users/aaa904c1683744de988c66ce350e8a6e/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.632192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4736c37dc824497ad9213da3a83d633, user_id=aaa904c1683744de988c66ce350e8a6e, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.634963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.661074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1103/2205] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:40 2022] PUT /identity/v3/projects/c4736c37dc824497ad9213da3a83d633/users/aaa904c1683744de988c66ce350e8a6e/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:40.664760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c644f9e-8624-4efd-bca3-f7099cc69840 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.666806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c644f9e-8624-4efd-bca3-f7099cc69840 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.667129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c644f9e-8624-4efd-bca3-f7099cc69840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.698911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0c644f9e-8624-4efd-bca3-f7099cc69840 None None] MFA Rules not processed for user `aaa904c1683744de988c66ce350e8a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:40.704179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0c644f9e-8624-4efd-bca3-f7099cc69840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.797968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1103/2206] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:40.809830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.812641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.890349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67946dda-90f8-4cfa-9230-f9c5d5fdc7b7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.899989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2f220-27ec-477f-9a1a-f3471801a6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.900742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f220-27ec-477f-9a1a-f3471801a6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.901107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f220-27ec-477f-9a1a-f3471801a6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.901447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f220-27ec-477f-9a1a-f3471801a6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.915401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:40.918397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2f220-27ec-477f-9a1a-f3471801a6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.920074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2f220-27ec-477f-9a1a-f3471801a6e1 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.932567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1104/2207] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:40.955175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c644f9e-8624-4efd-bca3-f7099cc69840 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.017117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:41.030105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c644f9e-8624-4efd-bca3-f7099cc69840 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.036599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.040778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.041260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.041546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] PATH_INFO: `/v3/users/aaa904c1683744de988c66ce350e8a6e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.043100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa904c1683744de988c66ce350e8a6e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.045900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.105978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa904c1683744de988c66ce350e8a6e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:41.106649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa904c1683744de988c66ce350e8a6e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:41.107259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa904c1683744de988c66ce350e8a6e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:41.118260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa904c1683744de988c66ce350e8a6e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:41.139705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa904c1683744de988c66ce350e8a6e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:41.158479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1104/2208] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:40 2022] DELETE /identity/v3/users/aaa904c1683744de988c66ce350e8a6e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:41.162990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.163275 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.163591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.192065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:41.193297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:41.261344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1105/2209] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:41.271579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.273220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:41.419209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea111e1-f78a-48a9-900d-5d4f9f2073b3 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.429385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8489836-48ae-4f95-a238-d56e13076ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.430020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8489836-48ae-4f95-a238-d56e13076ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.430298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8489836-48ae-4f95-a238-d56e13076ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.430568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8489836-48ae-4f95-a238-d56e13076ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.455610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8489836-48ae-4f95-a238-d56e13076ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.456526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8489836-48ae-4f95-a238-d56e13076ec6 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.474221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1105/2210] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:42.002491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:42.008128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:42.029183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c86f6-77d8-4019-a4c9-83bd59ab2cf7 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:42.044241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:42.045867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.046384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.046874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] PATH_INFO: `/v3/projects/c4736c37dc824497ad9213da3a83d633` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.048842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4736c37dc824497ad9213da3a83d633)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:42.051213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:42.144459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4736c37dc824497ad9213da3a83d633', 'request_id': 'req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:42.152182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4736c37dc824497ad9213da3a83d633', 'request_id': 'req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:42.159113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4736c37dc824497ad9213da3a83d633', 'request_id': 'req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:42.177726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] The token cache is being invalidate because project c4736c37dc824497ad9213da3a83d633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:42.177726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:42.179459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f9b62ab5-6a9d-441f-ba50-ff5c5a65645e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:42.184239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1106/2211] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:41 2022] DELETE /identity/v3/projects/c4736c37dc824497ad9213da3a83d633 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:42.186350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842d30f-03c3-4138-8562-c802120d41be None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.186784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842d30f-03c3-4138-8562-c802120d41be None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.187139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842d30f-03c3-4138-8562-c802120d41be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.216307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-7842d30f-03c3-4138-8562-c802120d41be None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:42.217855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7842d30f-03c3-4138-8562-c802120d41be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:42.296816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1106/2212] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:42.299292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179e03-8b4f-4aed-b647-08de70015d60 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.299566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179e03-8b4f-4aed-b647-08de70015d60 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.299877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179e03-8b4f-4aed-b647-08de70015d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.325767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-68179e03-8b4f-4aed-b647-08de70015d60 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:42.327097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-68179e03-8b4f-4aed-b647-08de70015d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:42.338174 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1107/2213] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:42.342681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842d30f-03c3-4138-8562-c802120d41be None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:42.353366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842d30f-03c3-4138-8562-c802120d41be None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:42.361742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:42.362728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.363007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.363275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.363974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:42.364280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:42.366028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:42.391006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1107/2214] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:42.392257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179e03-8b4f-4aed-b647-08de70015d60 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:42.405597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179e03-8b4f-4aed-b647-08de70015d60 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:42.412479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:42.413128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.413424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.417557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.418445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:42.420478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:42.478750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1108/2215] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:42 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:42.482864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:42.489588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:42.499275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569c3cc-050f-4b41-82d3-f34ebcc4aeb2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:42.625703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:42.627436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.627725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.628062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.628819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:42.631554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:42.667578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1108/2216] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:42 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:53:42.680730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:42.692864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ac2b3-a4cd-4305-9b20-b71a973e1a44 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:42.698049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:42.698668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.698945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.699214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.699869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:42.701501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:42.712514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1109/2217] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:42.716959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:42.730327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea116b1-0ca3-4f3f-8b2c-c46800fb4ffb admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:42.738782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f539e-ed8c-4e30-a650-9743dfc28099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:42.740134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f539e-ed8c-4e30-a650-9743dfc28099 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.740704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f539e-ed8c-4e30-a650-9743dfc28099 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.741304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f539e-ed8c-4e30-a650-9743dfc28099 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.742314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f539e-ed8c-4e30-a650-9743dfc28099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:42.745201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f539e-ed8c-4e30-a650-9743dfc28099 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:42.760558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1109/2218] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:42.764864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:42.775600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9323b-12d2-43e8-9dde-9791660d0fb5 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:42.780436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:42.781178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.781519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.781865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] PATH_INFO: `/v3/projects/cd7f0d6fb2ef49e99fff56d66ba5e7df/users/d0b0a7b3acb64e1caadafe7fa3383319/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.798135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7f0d6fb2ef49e99fff56d66ba5e7df, user_id=d0b0a7b3acb64e1caadafe7fa3383319, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:42.802371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:42.820695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1110/2219] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:42 2022] PUT /identity/v3/projects/cd7f0d6fb2ef49e99fff56d66ba5e7df/users/d0b0a7b3acb64e1caadafe7fa3383319/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:42.831783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:42.832285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:42.832721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:42.921270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] MFA Rules not processed for user `d0b0a7b3acb64e1caadafe7fa3383319`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:42.922216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:43.068945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1110/2220] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:43.085883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:43.087953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:43.123483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:43.142349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:43.143994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:43.199059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896179bc-435b-4648-9a6b-7c3b64a95e85 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:43.210282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5d376-e855-43db-b719-ca018cc5a817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:43.211109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5d376-e855-43db-b719-ca018cc5a817 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:43.211485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5d376-e855-43db-b719-ca018cc5a817 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:43.211847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5d376-e855-43db-b719-ca018cc5a817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:43.227405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5d376-e855-43db-b719-ca018cc5a817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:43.231574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5d376-e855-43db-b719-ca018cc5a817 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:43.240569 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094d80-9f51-4032-8520-4a80682b2627 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:43.250425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:43.251639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:43.252158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:43.252645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] PATH_INFO: `/v3/users/20ee1a29f5ef4d52adf45f926406b367` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:43.254495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ee1a29f5ef4d52adf45f926406b367)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:43.260140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:43.265838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1111/2221] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:43.296211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:43.370579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ee1a29f5ef4d52adf45f926406b367'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:43.371448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ee1a29f5ef4d52adf45f926406b367'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:43.372519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ee1a29f5ef4d52adf45f926406b367'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:43.381219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ee1a29f5ef4d52adf45f926406b367'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:43.415783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:43.416265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:43.416605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:43.417957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-482966a7-2935-4ba9-9b27-5cb29b5addca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ee1a29f5ef4d52adf45f926406b367'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:43.437491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1111/2222] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:43 2022] DELETE /identity/v3/users/20ee1a29f5ef4d52adf45f926406b367 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:43.442095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774f9fe-e268-4107-8845-45301b760595 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:43.442562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774f9fe-e268-4107-8845-45301b760595 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:43.442848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774f9fe-e268-4107-8845-45301b760595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:43.466051 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3774f9fe-e268-4107-8845-45301b760595 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:43.467306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3774f9fe-e268-4107-8845-45301b760595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:43.495063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] MFA Rules not processed for user `e71984f02fcd4deba5e1430ec85b8705`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:43.496928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:43.572627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1112/2223] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:43.579533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774f9fe-e268-4107-8845-45301b760595 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:43.581649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3774f9fe-e268-4107-8845-45301b760595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:43.630910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1112/2224] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:43.645371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:43.647272 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:43.673803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774f9fe-e268-4107-8845-45301b760595 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:43.677937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:43.678794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:43.679063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:43.679552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:43.702401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:43.708400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:43.775492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1113/2225] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:53:43.847724 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c51d5-80fe-4acb-a206-649be9f4ff7a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:43.854426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:43.854945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:43.855222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:43.855488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:43.872478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:43.872478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:43.886313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1113/2226] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:53:44.204872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:44.206563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:44.232790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5da7db-ff3f-4ccf-9068-f49b10937b0b service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:44.240942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:44.241599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:44.241904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:44.242168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] PATH_INFO: `/v3/projects/162de8ca3d3e40da9233bdb44aaf8ebe` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:44.245815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=162de8ca3d3e40da9233bdb44aaf8ebe)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:44.247746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:44.299312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:44.364914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162de8ca3d3e40da9233bdb44aaf8ebe', 'request_id': 'req-3c914178-053f-4c04-a6d4-7f5a3a036b8c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.372519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162de8ca3d3e40da9233bdb44aaf8ebe', 'request_id': 'req-3c914178-053f-4c04-a6d4-7f5a3a036b8c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.386060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162de8ca3d3e40da9233bdb44aaf8ebe', 'request_id': 'req-3c914178-053f-4c04-a6d4-7f5a3a036b8c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.409377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] The token cache is being invalidate because project 162de8ca3d3e40da9233bdb44aaf8ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:44.409377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.411801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.413496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1114/2227] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:44 2022] DELETE /identity/v3/projects/162de8ca3d3e40da9233bdb44aaf8ebe => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:44.417611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:44.429864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d42c9-f11d-4eae-abcc-2371cd559f50 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:44.477214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:44.512509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:44.513152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:44.513429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:44.513703 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] PATH_INFO: `/v3/users/2bb63d3526a34964a666b317f2523817` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:44.514857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb63d3526a34964a666b317f2523817)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:44.517086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:44.593299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb63d3526a34964a666b317f2523817'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.594001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb63d3526a34964a666b317f2523817'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.603826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb63d3526a34964a666b317f2523817'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.604627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb63d3526a34964a666b317f2523817'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.629199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-55c1c6a8-c21b-47cc-ae1c-b259d4b1e9a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb63d3526a34964a666b317f2523817'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:44.660026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1114/2228] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:44 2022] DELETE /identity/v3/users/2bb63d3526a34964a666b317f2523817 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:45.000217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.001812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:45.022385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c914178-053f-4c04-a6d4-7f5a3a036b8c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:45.129436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:45.130299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.130674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.131044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] PATH_INFO: `/v3/projects/07109830f0a8421186d7c6976e9a9b64` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.132422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07109830f0a8421186d7c6976e9a9b64)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:45.134769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:45.208599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07109830f0a8421186d7c6976e9a9b64', 'request_id': 'req-f36db449-e9ea-421d-a29e-c35745fc64e1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:45.236995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07109830f0a8421186d7c6976e9a9b64', 'request_id': 'req-f36db449-e9ea-421d-a29e-c35745fc64e1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:45.244969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07109830f0a8421186d7c6976e9a9b64', 'request_id': 'req-f36db449-e9ea-421d-a29e-c35745fc64e1'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:45.261104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] The token cache is being invalidate because project 07109830f0a8421186d7c6976e9a9b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:45.261399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:45.262121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f36db449-e9ea-421d-a29e-c35745fc64e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:45.270547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04debfe0-f21d-47a8-bb05-2c4e119a8d08 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.270897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04debfe0-f21d-47a8-bb05-2c4e119a8d08 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.271191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04debfe0-f21d-47a8-bb05-2c4e119a8d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.271489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1115/2229] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:44 2022] DELETE /identity/v3/projects/07109830f0a8421186d7c6976e9a9b64 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:45.304127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-04debfe0-f21d-47a8-bb05-2c4e119a8d08 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:45.305449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-04debfe0-f21d-47a8-bb05-2c4e119a8d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:45.384017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1115/2230] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:45.393421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3364a8-67e1-4040-aea0-4dda21a3dcc7 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.394484 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3364a8-67e1-4040-aea0-4dda21a3dcc7 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.394833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3364a8-67e1-4040-aea0-4dda21a3dcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.420633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5a3364a8-67e1-4040-aea0-4dda21a3dcc7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:45.421888 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5a3364a8-67e1-4040-aea0-4dda21a3dcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:45.440485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1116/2231] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:45.441271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04debfe0-f21d-47a8-bb05-2c4e119a8d08 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.452712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04debfe0-f21d-47a8-bb05-2c4e119a8d08 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:45.459448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:45.460122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.463676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.464034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.465089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:45.465674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:45.467510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:45.491161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1116/2232] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:45.492351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f377a7-1ede-47de-a695-5b5e27a3e62f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.492638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f377a7-1ede-47de-a695-5b5e27a3e62f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.492915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f377a7-1ede-47de-a695-5b5e27a3e62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.493518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.511456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73f0e6-24b6-40c3-85f1-09b855a11b0b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:45.516192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-a9f377a7-1ede-47de-a695-5b5e27a3e62f None None] MFA Rules not processed for user `e71984f02fcd4deba5e1430ec85b8705`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:45.517012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:45.517481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a9f377a7-1ede-47de-a695-5b5e27a3e62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:45.517772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.518120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.518726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.519650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:45.521755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:45.556000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1117/2233] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:45 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:45.559852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.560938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:45.584943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:45.588167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1117/2234] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:45.641767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2468a61-9cea-4e94-b77e-beaf46eae42f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:45.647088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:45.647706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.648019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.648310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.648928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:45.650865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:45.685694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1118/2235] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:45 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:53:45.690274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f377a7-1ede-47de-a695-5b5e27a3e62f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.704666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f377a7-1ede-47de-a695-5b5e27a3e62f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:45.712330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:45.715358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.715770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.716168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.716945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:45.719166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:45.731940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1118/2236] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:45.736192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.755212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a75112-b3aa-4cfe-9468-b04c044027a8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:45.760050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed9610-4db7-4cb5-92e7-844cb9fb4a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:45.760875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9610-4db7-4cb5-92e7-844cb9fb4a4f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.761230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9610-4db7-4cb5-92e7-844cb9fb4a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.761623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9610-4db7-4cb5-92e7-844cb9fb4a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.762572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed9610-4db7-4cb5-92e7-844cb9fb4a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:45.764536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed9610-4db7-4cb5-92e7-844cb9fb4a4f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:45.777557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1119/2237] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:45.781959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.794895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f953e-263d-4bb1-9aef-81f97dc49356 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:45.800102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:45.800664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.800950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.801283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] PATH_INFO: `/v3/projects/ddec34749c144b2aa1592047ec3a6b3b/users/21a3ea87ec4c44218897872be8184f09/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.822447 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddec34749c144b2aa1592047ec3a6b3b, user_id=21a3ea87ec4c44218897872be8184f09, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:45.826831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:45.846516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1119/2238] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:45 2022] PUT /identity/v3/projects/ddec34749c144b2aa1592047ec3a6b3b/users/21a3ea87ec4c44218897872be8184f09/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:53:45.857127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fb901-ee5a-465a-80a8-d6f80600b1cf None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:45.857496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fb901-ee5a-465a-80a8-d6f80600b1cf None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:45.857839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fb901-ee5a-465a-80a8-d6f80600b1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:45.888128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-294fb901-ee5a-465a-80a8-d6f80600b1cf None None] MFA Rules not processed for user `21a3ea87ec4c44218897872be8184f09`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:45.889369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-294fb901-ee5a-465a-80a8-d6f80600b1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:45.964628 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1120/2239] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:53:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:45.979554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:45.980560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:46.056021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d41-e780-4f2e-b3a3-fd82fd6807db admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:46.060701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf70040-3059-4a3b-b4f9-37fc36cf9bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:46.061317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf70040-3059-4a3b-b4f9-37fc36cf9bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:46.061597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf70040-3059-4a3b-b4f9-37fc36cf9bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:46.061870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf70040-3059-4a3b-b4f9-37fc36cf9bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:46.072439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf70040-3059-4a3b-b4f9-37fc36cf9bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:46.073894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf70040-3059-4a3b-b4f9-37fc36cf9bd0 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:46.085683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1120/2240] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:55.959042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:56.079478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=48,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:01.036634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fb901-ee5a-465a-80a8-d6f80600b1cf None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:01.103517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fb901-ee5a-465a-80a8-d6f80600b1cf None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:01.109013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:01.109316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:01.109550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:01.109934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] PATH_INFO: `/v3/users/21a3ea87ec4c44218897872be8184f09` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:01.111012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a3ea87ec4c44218897872be8184f09)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:01.113498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:01.184647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a3ea87ec4c44218897872be8184f09'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:01.185523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a3ea87ec4c44218897872be8184f09'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:01.196935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a3ea87ec4c44218897872be8184f09'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:01.198224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a3ea87ec4c44218897872be8184f09'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:01.228339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a3ea87ec4c44218897872be8184f09'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:01.259626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1121/2241] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:01 2022] DELETE /identity/v3/users/21a3ea87ec4c44218897872be8184f09 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:01.264354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:01.264912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:01.264912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:01.287920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:01.289302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:01.386307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1121/2242] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:01.645327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:01.646841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:01.744692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec33f5d-132a-4ac2-a530-a561b280ffa4 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:01.749405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6509aa-f42d-4914-9b2a-ac0b5d459fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:01.750034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6509aa-f42d-4914-9b2a-ac0b5d459fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:01.750335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6509aa-f42d-4914-9b2a-ac0b5d459fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:01.750575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6509aa-f42d-4914-9b2a-ac0b5d459fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:01.769557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6509aa-f42d-4914-9b2a-ac0b5d459fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:01.771017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6509aa-f42d-4914-9b2a-ac0b5d459fd5 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:01.781420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1122/2243] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:02.200678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:02.202748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:02.216655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69d101-f9a0-465b-8279-b2ea60ef895f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:02.222502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:02.223145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.223397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.223694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] PATH_INFO: `/v3/projects/ddec34749c144b2aa1592047ec3a6b3b` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.224970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddec34749c144b2aa1592047ec3a6b3b)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:02.226883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:02.287942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddec34749c144b2aa1592047ec3a6b3b', 'request_id': 'req-bc85408a-e94a-4d51-bb86-62d8456f638b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:02.301243 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddec34749c144b2aa1592047ec3a6b3b', 'request_id': 'req-bc85408a-e94a-4d51-bb86-62d8456f638b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:02.308181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddec34749c144b2aa1592047ec3a6b3b', 'request_id': 'req-bc85408a-e94a-4d51-bb86-62d8456f638b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:02.334144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] The token cache is being invalidate because project ddec34749c144b2aa1592047ec3a6b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:02.334482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:02.335151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-bc85408a-e94a-4d51-bb86-62d8456f638b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:02.336778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1122/2244] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:02 2022] DELETE /identity/v3/projects/ddec34749c144b2aa1592047ec3a6b3b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:02.341655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56eae1-5cc0-4ba3-95bb-8d4278fe5be8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.342147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56eae1-5cc0-4ba3-95bb-8d4278fe5be8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.342254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56eae1-5cc0-4ba3-95bb-8d4278fe5be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.364188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9e56eae1-5cc0-4ba3-95bb-8d4278fe5be8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:02.365395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9e56eae1-5cc0-4ba3-95bb-8d4278fe5be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:02.395638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:02.458831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1123/2245] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:02.460749 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6dc0-b149-46fe-a509-3bea79b70c10 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.461548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6dc0-b149-46fe-a509-3bea79b70c10 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.461548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f6dc0-b149-46fe-a509-3bea79b70c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.490064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b42f6dc0-b149-46fe-a509-3bea79b70c10 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:02.491332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b42f6dc0-b149-46fe-a509-3bea79b70c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:02.499452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1123/2246] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:02.503647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56eae1-5cc0-4ba3-95bb-8d4278fe5be8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:02.514500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56eae1-5cc0-4ba3-95bb-8d4278fe5be8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:02.519547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:02.520664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.521048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.521425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.522320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:02.522839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:02.526257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:02.546001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1124/2247] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:02.550086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f6dc0-b149-46fe-a509-3bea79b70c10 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:02.560188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f6dc0-b149-46fe-a509-3bea79b70c10 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:02.565029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:02.565864 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.566155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.567097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.567761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:02.570050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:02.621360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1124/2248] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:02 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:02.625749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:02.637118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a124-6e76-4ffd-a040-f573399c7bdd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:02.708380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:02.709010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.709343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.709742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.710481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:02.712534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:02.771483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1125/2249] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:02 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:02.775593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:02.788057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302b5e2-4a2c-4d1a-8080-ea8391d6e789 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:02.796795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:02.797435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.797715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.797994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.798666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:02.800315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:02.819007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1125/2250] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:02.820713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:02.845542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d9c5a-849a-4f51-93a9-af5e0690d3e2 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:02.852560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6823eb0-8d9c-41d9-aca7-9935ca1c8f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:02.853439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6823eb0-8d9c-41d9-aca7-9935ca1c8f32 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.853890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6823eb0-8d9c-41d9-aca7-9935ca1c8f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.854340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6823eb0-8d9c-41d9-aca7-9935ca1c8f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.855046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6823eb0-8d9c-41d9-aca7-9935ca1c8f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:02.857385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6823eb0-8d9c-41d9-aca7-9935ca1c8f32 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:02.894688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1126/2251] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:02.895772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:02.907913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfd983-ba53-4b62-b593-34afd676a28f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:02.914935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:02.915770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.916196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.916804 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] PATH_INFO: `/v3/projects/8d755979454645268767c74f2674c482/users/92cea8cd1a364adca5ad165183718ef0/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:02.948665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d755979454645268767c74f2674c482, user_id=92cea8cd1a364adca5ad165183718ef0, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:02.951574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:02.980047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1126/2252] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:02 2022] PUT /identity/v3/projects/8d755979454645268767c74f2674c482/users/92cea8cd1a364adca5ad165183718ef0/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:02.991340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c244-339a-4002-8471-5465ee300460 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:02.991742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c244-339a-4002-8471-5465ee300460 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:02.992198 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c244-339a-4002-8471-5465ee300460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:03.026592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-4f21c244-339a-4002-8471-5465ee300460 None None] MFA Rules not processed for user `92cea8cd1a364adca5ad165183718ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:03.028031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4f21c244-339a-4002-8471-5465ee300460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:03.106053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1127/2253] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:03.119498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:03.130446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:03.155992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:03.199122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2eba75-45f3-4d3e-b6ec-7b089bb9523c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:03.202955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dc195-8f27-4443-915a-23ce20569ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:03.203567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dc195-8f27-4443-915a-23ce20569ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:03.203841 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dc195-8f27-4443-915a-23ce20569ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:03.204161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dc195-8f27-4443-915a-23ce20569ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:03.213860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dc195-8f27-4443-915a-23ce20569ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:03.216518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dc195-8f27-4443-915a-23ce20569ca5 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:03.226231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1127/2254] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:03.254543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c244-339a-4002-8471-5465ee300460 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:03.277994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:03.329850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c244-339a-4002-8471-5465ee300460 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:03.335000 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:03.335655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:03.335964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:03.336239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] PATH_INFO: `/v3/users/92cea8cd1a364adca5ad165183718ef0` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:03.337390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92cea8cd1a364adca5ad165183718ef0)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:03.339404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:03.404908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92cea8cd1a364adca5ad165183718ef0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:03.405876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92cea8cd1a364adca5ad165183718ef0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:03.415034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92cea8cd1a364adca5ad165183718ef0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:03.416391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92cea8cd1a364adca5ad165183718ef0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:03.440836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92cea8cd1a364adca5ad165183718ef0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:03.461045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1128/2255] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:03 2022] DELETE /identity/v3/users/92cea8cd1a364adca5ad165183718ef0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:03.465247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:03.466127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:03.478227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:03.486458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:03.488115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:03.557983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1128/2256] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:03.568139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:03.569648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:03.658949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b695ef-f7a4-4ff7-9694-47d139bd1645 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:03.665817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78363d-8687-4054-9604-c4bb27e73f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:03.666620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78363d-8687-4054-9604-c4bb27e73f63 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:03.666986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78363d-8687-4054-9604-c4bb27e73f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:03.667382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78363d-8687-4054-9604-c4bb27e73f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:03.681586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78363d-8687-4054-9604-c4bb27e73f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:03.683097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78363d-8687-4054-9604-c4bb27e73f63 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:03.694139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1129/2257] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:54:04.004913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.006476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:04.029736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af53f7-2573-446d-b090-2760e6ed2d70 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.035770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.036618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.037005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.037356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] PATH_INFO: `/v3/projects/8d755979454645268767c74f2674c482` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.038721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d755979454645268767c74f2674c482)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.040779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.097409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d755979454645268767c74f2674c482', 'request_id': 'req-b905d437-3756-439c-b04f-2f26e9e45379'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:04.106083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d755979454645268767c74f2674c482', 'request_id': 'req-b905d437-3756-439c-b04f-2f26e9e45379'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:04.116328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d755979454645268767c74f2674c482', 'request_id': 'req-b905d437-3756-439c-b04f-2f26e9e45379'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:04.132896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] The token cache is being invalidate because project 8d755979454645268767c74f2674c482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:04.133196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:04.133958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b905d437-3756-439c-b04f-2f26e9e45379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:04.135708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1129/2258] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:03 2022] DELETE /identity/v3/projects/8d755979454645268767c74f2674c482 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:04.140419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4645-45ab-4d83-b22e-dc79dbd23376 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.140780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4645-45ab-4d83-b22e-dc79dbd23376 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.141141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4645-45ab-4d83-b22e-dc79dbd23376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.164076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a29c4645-45ab-4d83-b22e-dc79dbd23376 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:04.165432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a29c4645-45ab-4d83-b22e-dc79dbd23376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:04.214726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:04.250103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1130/2259] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:04.259849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2209b48-4283-4091-882b-dd1781b932ea None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.260201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2209b48-4283-4091-882b-dd1781b932ea None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.260477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2209b48-4283-4091-882b-dd1781b932ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.282032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d2209b48-4283-4091-882b-dd1781b932ea None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:04.283277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d2209b48-4283-4091-882b-dd1781b932ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:04.291235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1130/2260] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:04.296074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4645-45ab-4d83-b22e-dc79dbd23376 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.306374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4645-45ab-4d83-b22e-dc79dbd23376 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.311139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.311749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.312063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.312336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.312996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:04.313303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.314892 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.331585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1131/2261] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:04.338304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2209b48-4283-4091-882b-dd1781b932ea None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.350514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2209b48-4283-4091-882b-dd1781b932ea None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.355456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.356100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.356382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.356654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.357329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.359233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.389997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1131/2262] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:04 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:04.394776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.406065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fe828-e369-4f4e-8674-530b1cd7bb8c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.466840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.467358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.467634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.467914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.468548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.470478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.502614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1132/2263] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:04 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:04.503878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.515299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c6716-1f7a-4f62-8ffd-b55c619866a6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.520574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.521211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.521481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.521827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.522526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.524207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.535915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1132/2264] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:04.538406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.548541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1dd64-1d3f-4cd4-b71c-8395ed4e871b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.553546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a672f1e-b4ad-4a2a-9284-1fc31abf2393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.554146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672f1e-b4ad-4a2a-9284-1fc31abf2393 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.554425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672f1e-b4ad-4a2a-9284-1fc31abf2393 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.554671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a672f1e-b4ad-4a2a-9284-1fc31abf2393 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.555333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a672f1e-b4ad-4a2a-9284-1fc31abf2393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.556991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a672f1e-b4ad-4a2a-9284-1fc31abf2393 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.567912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1133/2265] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:04.572037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.581895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4d700-66a4-412e-b5a4-6a3e7f419542 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.586726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.587403 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.587684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.587992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] PATH_INFO: `/v3/projects/67b474ba7f7a465caa2ed14f891ac951/users/1e44ee032c5346ec9260740bcb700007/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.590487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:04.605031 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b474ba7f7a465caa2ed14f891ac951, user_id=1e44ee032c5346ec9260740bcb700007, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.607688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.622950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1133/2266] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:04 2022] PUT /identity/v3/projects/67b474ba7f7a465caa2ed14f891ac951/users/1e44ee032c5346ec9260740bcb700007/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:04.628044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d2a3e-d0be-4592-92d4-70dc6a53b938 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.628331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d2a3e-d0be-4592-92d4-70dc6a53b938 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.628603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d2a3e-d0be-4592-92d4-70dc6a53b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.647300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-718d2a3e-d0be-4592-92d4-70dc6a53b938 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:04.648534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-718d2a3e-d0be-4592-92d4-70dc6a53b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:04.720256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1134/2267] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:04.729535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:04.731760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:04.796052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f06a7c-2c37-42d1-ac1a-0c0a743dfc2b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:04.799941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:04.800550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:04.800836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:04.801100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:04.810119 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:04.811542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:04.823062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1134/2268] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:05.690078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40b01-438c-4760-bb75-6f9e37f54fd1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:05.690644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40b01-438c-4760-bb75-6f9e37f54fd1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:05.690644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40b01-438c-4760-bb75-6f9e37f54fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:05.719780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6fc40b01-438c-4760-bb75-6f9e37f54fd1 None None] MFA Rules not processed for user `1e44ee032c5346ec9260740bcb700007`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:05.721009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6fc40b01-438c-4760-bb75-6f9e37f54fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:05.765045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:05.805533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1135/2269] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:05.817850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:05.822311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:05.880778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903c0ad-7dd9-4f57-a6cf-2eb5a6e8e841 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:05.884856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:05.885485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:05.885764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:05.886033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:05.895625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:05.897236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:05.905822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1135/2270] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:11.534777 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed01b1-8fd3-4f9b-88d8-4ce4243446f5 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:11.535248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed01b1-8fd3-4f9b-88d8-4ce4243446f5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:11.535362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed01b1-8fd3-4f9b-88d8-4ce4243446f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:11.554713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-10ed01b1-8fd3-4f9b-88d8-4ce4243446f5 None None] MFA Rules not processed for user `1e44ee032c5346ec9260740bcb700007`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:11.555879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-10ed01b1-8fd3-4f9b-88d8-4ce4243446f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:11.563255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1136/2271] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:15.424300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:15.425832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:15.439623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5d1a5-bb79-4332-a9c9-65194df4e872 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:15.444144 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:15.444769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:15.445087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:15.445357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] PATH_INFO: `/v3/users/d0b0a7b3acb64e1caadafe7fa3383319` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:15.446432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b0a7b3acb64e1caadafe7fa3383319)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:15.448374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:15.502507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b0a7b3acb64e1caadafe7fa3383319'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:15.503204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b0a7b3acb64e1caadafe7fa3383319'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:15.503810 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b0a7b3acb64e1caadafe7fa3383319'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:15.515584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b0a7b3acb64e1caadafe7fa3383319'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:15.532846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b0a7b3acb64e1caadafe7fa3383319'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:15.544446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:15.549004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1136/2272] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:15 2022] DELETE /identity/v3/users/d0b0a7b3acb64e1caadafe7fa3383319 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:15.553987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:15.554488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:15.554852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:15.575030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:15.576321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:15.641345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1137/2273] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:15.652870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:15.654420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:15.716376 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd024b7-0fea-49ed-b30c-766f74891820 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:15.720423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b49884-ff39-499b-9cfe-3cc3d982911f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:15.721062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b49884-ff39-499b-9cfe-3cc3d982911f service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:15.721339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b49884-ff39-499b-9cfe-3cc3d982911f service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:15.721607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b49884-ff39-499b-9cfe-3cc3d982911f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:15.730869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b49884-ff39-499b-9cfe-3cc3d982911f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:15.732359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b49884-ff39-499b-9cfe-3cc3d982911f service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:15.743923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1137/2274] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:16.032712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.034468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:16.047323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef67c8-b732-41e6-b24f-74b8c97626fc None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.052117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.052826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.053178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.053510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] PATH_INFO: `/v3/projects/cd7f0d6fb2ef49e99fff56d66ba5e7df` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.054666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7f0d6fb2ef49e99fff56d66ba5e7df)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.056615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.113447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7f0d6fb2ef49e99fff56d66ba5e7df', 'request_id': 'req-fe1c4183-41f8-468b-ab56-f835e2060ba1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:16.119771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7f0d6fb2ef49e99fff56d66ba5e7df', 'request_id': 'req-fe1c4183-41f8-468b-ab56-f835e2060ba1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:16.125806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7f0d6fb2ef49e99fff56d66ba5e7df', 'request_id': 'req-fe1c4183-41f8-468b-ab56-f835e2060ba1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:16.149226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] The token cache is being invalidate because project cd7f0d6fb2ef49e99fff56d66ba5e7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:16.149495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:16.150155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fe1c4183-41f8-468b-ab56-f835e2060ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:16.151917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1138/2275] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:16 2022] DELETE /identity/v3/projects/cd7f0d6fb2ef49e99fff56d66ba5e7df => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:16.158956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e224f28-c3a9-4578-842b-1f8b06e7470e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.159477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e224f28-c3a9-4578-842b-1f8b06e7470e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.159974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e224f28-c3a9-4578-842b-1f8b06e7470e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.182660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5e224f28-c3a9-4578-842b-1f8b06e7470e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:16.183707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5e224f28-c3a9-4578-842b-1f8b06e7470e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:16.265492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1138/2276] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:16.273645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5cc78-c643-4b56-a802-186cd7d7bfd8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.273930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5cc78-c643-4b56-a802-186cd7d7bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.274212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5cc78-c643-4b56-a802-186cd7d7bfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.296489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-adc5cc78-c643-4b56-a802-186cd7d7bfd8 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:16.297748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-adc5cc78-c643-4b56-a802-186cd7d7bfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:16.306301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1139/2277] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:16.311333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e224f28-c3a9-4578-842b-1f8b06e7470e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.324396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e224f28-c3a9-4578-842b-1f8b06e7470e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.329705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.330319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.330651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.330915 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.331576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:16.331929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.333480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.345964 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1139/2278] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:16.353930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5cc78-c643-4b56-a802-186cd7d7bfd8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.360120 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5cc78-c643-4b56-a802-186cd7d7bfd8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.365866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.366462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.366749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.367020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.367651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.369685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.399459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1140/2279] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:16 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:16.404027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.415730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6cef8-bf8d-44fc-8fe9-a49fe1e54865 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.481532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.482221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.482523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.482796 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.483431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.485444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.511091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1140/2280] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:16 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:16.515660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.531469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f87bc-5e34-482d-85d2-6942bb6523fb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.537893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.538494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.538806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.539070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.539719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.541786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.553631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1141/2281] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:16.557817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.568047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f31f9-64be-45ca-bafb-ed37da10053c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.573231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0715e1-fa91-4dde-afe7-2b50224386df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.573838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0715e1-fa91-4dde-afe7-2b50224386df admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.574115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0715e1-fa91-4dde-afe7-2b50224386df admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.574386 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0715e1-fa91-4dde-afe7-2b50224386df admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.575163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0715e1-fa91-4dde-afe7-2b50224386df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.576771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0715e1-fa91-4dde-afe7-2b50224386df admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.586904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1141/2282] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:16.591104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.601810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435645b-ee9d-4ecd-9d67-58dab53798ed admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.607322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.608142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.608443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.608709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] PATH_INFO: `/v3/projects/19f97006db3c4c2494539225cf1f14bd/users/010ce216a05b4222b473934619aaba8e/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.621711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:16.626419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f97006db3c4c2494539225cf1f14bd, user_id=010ce216a05b4222b473934619aaba8e, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.629784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.645121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1142/2283] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:16 2022] PUT /identity/v3/projects/19f97006db3c4c2494539225cf1f14bd/users/010ce216a05b4222b473934619aaba8e/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:16.655499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18c0d6-bbb7-4d37-8063-561b9dd41096 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.655726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18c0d6-bbb7-4d37-8063-561b9dd41096 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.656045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18c0d6-bbb7-4d37-8063-561b9dd41096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.685033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:16.696722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-8f18c0d6-bbb7-4d37-8063-561b9dd41096 None None] MFA Rules not processed for user `010ce216a05b4222b473934619aaba8e`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:16.698366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8f18c0d6-bbb7-4d37-8063-561b9dd41096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:16.763976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1142/2284] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:16.768855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.837619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075d0ec-3e79-427e-b0d5-8e3e2ce3ef90 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.842588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.843209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.843484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.843750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.844417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.846306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.875691 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1143/2285] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:16 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:16.879778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18c0d6-bbb7-4d37-8063-561b9dd41096 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:16.891032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18c0d6-bbb7-4d37-8063-561b9dd41096 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:16.957539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:16.958159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:16.958476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:16.958747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:16.959388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:16.961358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:16.987946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1143/2286] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:16 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:16.991224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.002291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2659c-7672-4cb0-a128-65313b8ed6c0 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.007133 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.007747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.008113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.008446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.009288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.011067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.021946 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1144/2287] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:17.026267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.036338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe5718-d3eb-4f01-a54d-bea2b4d78cea admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.041300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eb246-e139-480f-83de-baafcf7a607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.041958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb246-e139-480f-83de-baafcf7a607b admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.042220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb246-e139-480f-83de-baafcf7a607b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.042486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb246-e139-480f-83de-baafcf7a607b admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.043140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1eb246-e139-480f-83de-baafcf7a607b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.044769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1eb246-e139-480f-83de-baafcf7a607b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.054626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1144/2288] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:17.058497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.068535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d47ba-79ec-4313-b3bb-6e18c96555bd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.073359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.074004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.074315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.074563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] PATH_INFO: `/v3/projects/9ed8feeeec09411fb4814ec274c43caf/users/465873f4cf514e4986bdc93dbb949499/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.091625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed8feeeec09411fb4814ec274c43caf, user_id=465873f4cf514e4986bdc93dbb949499, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.094339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.109154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1145/2289] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:17 2022] PUT /identity/v3/projects/9ed8feeeec09411fb4814ec274c43caf/users/465873f4cf514e4986bdc93dbb949499/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:17.121206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d055-77c8-465d-bdb5-2aa89df05f8e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.121493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d055-77c8-465d-bdb5-2aa89df05f8e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.121792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d055-77c8-465d-bdb5-2aa89df05f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.156235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4321d055-77c8-465d-bdb5-2aa89df05f8e None None] MFA Rules not processed for user `465873f4cf514e4986bdc93dbb949499`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:17.157531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4321d055-77c8-465d-bdb5-2aa89df05f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:17.225642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1145/2290] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:17.229973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.307201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570efd47-582c-49cf-81a1-a84853725644 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.312401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.313027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.313304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.313573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.314226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.316190 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.348849 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1146/2291] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:17 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:17.350104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321d055-77c8-465d-bdb5-2aa89df05f8e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.360975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321d055-77c8-465d-bdb5-2aa89df05f8e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.364855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:17.429519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.430132 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.430412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.430655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.431445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.433462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.460638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1146/2292] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:17 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:17.465106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.481331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11c5e6-2826-4a5c-9fc6-2d60014f51da admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.486548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.487150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.487398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.488050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.488694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.490293 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.502785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1147/2293] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:17.507549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.518607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c8403-43b3-45fc-8991-e2378ce06458 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.528213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.528940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.529226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.529518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] PATH_INFO: `/v3/projects/736ef148be0e46a6a5d6f70545f6fc09/users/e4ca41eb2bc34df79561735a1608f04b/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.545648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736ef148be0e46a6a5d6f70545f6fc09, user_id=e4ca41eb2bc34df79561735a1608f04b, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.548337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.563305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1147/2294] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:17 2022] PUT /identity/v3/projects/736ef148be0e46a6a5d6f70545f6fc09/users/e4ca41eb2bc34df79561735a1608f04b/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:17.567307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.577589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d359f9d-ba2c-476e-bea9-97ae91be1345 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.586278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:17.638966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e6a80-033a-493e-bf6e-0d26b0eb4367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.639643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e6a80-033a-493e-bf6e-0d26b0eb4367 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.639852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e6a80-033a-493e-bf6e-0d26b0eb4367 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.640158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e6a80-033a-493e-bf6e-0d26b0eb4367 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.640802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e6a80-033a-493e-bf6e-0d26b0eb4367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.642396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e6a80-033a-493e-bf6e-0d26b0eb4367 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.655381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1148/2295] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:17.656951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.668521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1a749-540e-4dcd-a1ea-56398f2af473 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.673640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.674322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.674592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.674857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] PATH_INFO: `/v3/projects/736ef148be0e46a6a5d6f70545f6fc09/users/e4ca41eb2bc34df79561735a1608f04b/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.677180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736ef148be0e46a6a5d6f70545f6fc09, user_id=e4ca41eb2bc34df79561735a1608f04b, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.679825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.690850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1148/2296] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:17 2022] PUT /identity/v3/projects/736ef148be0e46a6a5d6f70545f6fc09/users/e4ca41eb2bc34df79561735a1608f04b/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:17.701033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63321b65-474f-42a3-9563-db73e318e27c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.701316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63321b65-474f-42a3-9563-db73e318e27c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.701585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63321b65-474f-42a3-9563-db73e318e27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.732191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-63321b65-474f-42a3-9563-db73e318e27c None None] MFA Rules not processed for user `e4ca41eb2bc34df79561735a1608f04b`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:17.733391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-63321b65-474f-42a3-9563-db73e318e27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:17.807018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1149/2297] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:17.818874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:17.820908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:17.884224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcbf1a-9c49-423d-899c-73d27586c6d5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:17.888343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:17.888950 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:17.889223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:17.889483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:17.952798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:17.954285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:17.963486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1149/2298] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:19.522182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63321b65-474f-42a3-9563-db73e318e27c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:19.523785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-63321b65-474f-42a3-9563-db73e318e27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:19.548184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63321b65-474f-42a3-9563-db73e318e27c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:19.548184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:19.548184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:19.548184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:19.548751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:19.608938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:19.610421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:19.620053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1150/2299] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:20.212355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:20.221404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604150-aa2f-441e-9ac8-f09d1defe49e service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:20.225343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac8fbd-813c-4efd-86ef-a431b350812a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:20.225952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac8fbd-813c-4efd-86ef-a431b350812a service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:20.226233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac8fbd-813c-4efd-86ef-a431b350812a service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:20.226502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac8fbd-813c-4efd-86ef-a431b350812a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:20.235928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac8fbd-813c-4efd-86ef-a431b350812a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:20.237364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac8fbd-813c-4efd-86ef-a431b350812a service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:20.242641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:20.264464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1150/2300] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:25.274752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:25.321263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:25.360448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99bbe6-8878-451b-ade2-d452654d7299 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:25.370305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:25.370973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:25.371252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:25.371523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] PATH_INFO: `/v3/users/010ce216a05b4222b473934619aaba8e` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:25.372930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=010ce216a05b4222b473934619aaba8e)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:25.375171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:25.438342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010ce216a05b4222b473934619aaba8e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:25.439260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010ce216a05b4222b473934619aaba8e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:25.446612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010ce216a05b4222b473934619aaba8e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:25.447276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010ce216a05b4222b473934619aaba8e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:25.459439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010ce216a05b4222b473934619aaba8e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:25.475734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1151/2301] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:25 2022] DELETE /identity/v3/users/010ce216a05b4222b473934619aaba8e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:25.481172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:25.481742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:25.483774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:25.508801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:25.510075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:25.602409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1151/2302] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:25.622200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:25.624023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:25.701687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8869d2-a8b6-475b-a37f-55e8f6afa5f8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:25.706304 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:25.706933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:25.707206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:25.707608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:25.717871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:25.719395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:25.729420 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1152/2303] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:26.017531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:26.019313 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:26.041463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464283aa-4c1b-4895-9275-cf3a879750b0 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:26.046167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:26.046746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:26.047028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:26.047305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] PATH_INFO: `/v3/projects/19f97006db3c4c2494539225cf1f14bd` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:26.048463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f97006db3c4c2494539225cf1f14bd)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:26.050401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:26.099474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f97006db3c4c2494539225cf1f14bd', 'request_id': 'req-dbf9acb2-290b-47c7-a70b-eda5185e2837'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.107017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f97006db3c4c2494539225cf1f14bd', 'request_id': 'req-dbf9acb2-290b-47c7-a70b-eda5185e2837'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.116967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f97006db3c4c2494539225cf1f14bd', 'request_id': 'req-dbf9acb2-290b-47c7-a70b-eda5185e2837'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.134974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] The token cache is being invalidate because project 19f97006db3c4c2494539225cf1f14bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:26.135260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.135921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.139957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1152/2304] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:26 2022] DELETE /identity/v3/projects/19f97006db3c4c2494539225cf1f14bd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:26.141666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:26.151242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1458219-3ea7-44de-aecc-a3421edf8296 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:26.202653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:26.211034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:26.211720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:26.212009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:26.212288 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] PATH_INFO: `/v3/users/465873f4cf514e4986bdc93dbb949499` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:26.213654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=465873f4cf514e4986bdc93dbb949499)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:26.216047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:26.265331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465873f4cf514e4986bdc93dbb949499'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.267099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465873f4cf514e4986bdc93dbb949499'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.274237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465873f4cf514e4986bdc93dbb949499'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.274971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465873f4cf514e4986bdc93dbb949499'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.303859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465873f4cf514e4986bdc93dbb949499'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:26.340733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1153/2305] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:26 2022] DELETE /identity/v3/users/465873f4cf514e4986bdc93dbb949499 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:26.831582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:26.835302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:26.867850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9acb2-290b-47c7-a70b-eda5185e2837 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:26.996197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:26.996960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:26.997236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:26.997911 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] PATH_INFO: `/v3/projects/9ed8feeeec09411fb4814ec274c43caf` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:26.999278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed8feeeec09411fb4814ec274c43caf)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:27.001477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:27.047833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:27.073130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed8feeeec09411fb4814ec274c43caf', 'request_id': 'req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.080303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed8feeeec09411fb4814ec274c43caf', 'request_id': 'req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.085995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed8feeeec09411fb4814ec274c43caf', 'request_id': 'req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.100398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] The token cache is being invalidate because project 9ed8feeeec09411fb4814ec274c43caf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:27.100698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.101297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.102799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1153/2306] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:26 2022] DELETE /identity/v3/projects/9ed8feeeec09411fb4814ec274c43caf => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:27.106985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:27.108567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:27.122221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa66c-1866-4026-838d-075668b441a9 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:27.180629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:27.181425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:27.181710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:27.181993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] PATH_INFO: `/v3/users/e4ca41eb2bc34df79561735a1608f04b` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:27.183155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ca41eb2bc34df79561735a1608f04b)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:27.185324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:27.245389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ca41eb2bc34df79561735a1608f04b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.245941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ca41eb2bc34df79561735a1608f04b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.253178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ca41eb2bc34df79561735a1608f04b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.253793 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ca41eb2bc34df79561735a1608f04b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.270679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ca41eb2bc34df79561735a1608f04b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.273292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:27.290061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1154/2307] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:27 2022] DELETE /identity/v3/users/e4ca41eb2bc34df79561735a1608f04b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:27.663351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:27.664877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:27.689107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb8ff2-0595-49ad-b70e-94e8f878cc85 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:27.897694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:27.898427 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:27.898775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:27.899055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] PATH_INFO: `/v3/projects/736ef148be0e46a6a5d6f70545f6fc09` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:27.900258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=736ef148be0e46a6a5d6f70545f6fc09)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:27.902214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:27.949461 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736ef148be0e46a6a5d6f70545f6fc09', 'request_id': 'req-57b737b3-4359-4058-9603-e8e44d178142'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.955693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736ef148be0e46a6a5d6f70545f6fc09', 'request_id': 'req-57b737b3-4359-4058-9603-e8e44d178142'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.961376 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736ef148be0e46a6a5d6f70545f6fc09', 'request_id': 'req-57b737b3-4359-4058-9603-e8e44d178142'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.979638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] The token cache is being invalidate because project 736ef148be0e46a6a5d6f70545f6fc09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:27.979963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.980618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.982404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1154/2308] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:27 2022] DELETE /identity/v3/projects/736ef148be0e46a6a5d6f70545f6fc09 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:37.291100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:37.977899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:53.449536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:53.451192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:53.522374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387aae3-ef40-4a5a-8314-292dba47bd85 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:53.589810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e264f07-e7dd-4ef8-bc7a-58fc58633dd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:53.590553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e264f07-e7dd-4ef8-bc7a-58fc58633dd9 service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:53.590830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e264f07-e7dd-4ef8-bc7a-58fc58633dd9 service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:53.591098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e264f07-e7dd-4ef8-bc7a-58fc58633dd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:53.592985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4e264f07-e7dd-4ef8-bc7a-58fc58633dd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:53.605027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e264f07-e7dd-4ef8-bc7a-58fc58633dd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:53.607374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e264f07-e7dd-4ef8-bc7a-58fc58633dd9 service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:53.619699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1155/2309] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:54.245711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:54.247388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:54.318205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b737b3-4359-4058-9603-e8e44d178142 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:54.323117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:54.323743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:54.324085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:54.324396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] PATH_INFO: `/v3/users/e56394ef9f144ebf9e8b8bc3f758bacc` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:54.325649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56394ef9f144ebf9e8b8bc3f758bacc)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:54.327548 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:54.381200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56394ef9f144ebf9e8b8bc3f758bacc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:54.381967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56394ef9f144ebf9e8b8bc3f758bacc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:54.382652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56394ef9f144ebf9e8b8bc3f758bacc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:54.390339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56394ef9f144ebf9e8b8bc3f758bacc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:54.402093 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56394ef9f144ebf9e8b8bc3f758bacc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:54.416839 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1155/2310] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:54 2022] DELETE /identity/v3/users/e56394ef9f144ebf9e8b8bc3f758bacc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:54.421739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:54.422020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:54.422313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:54.440875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:54.442071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:54.506250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1156/2311] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:54.516491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:54.517974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:54.579270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25469a92-e0e5-4910-860a-a7c11b5ae5c5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:54.583008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93a752-38c3-4740-8712-04b9d772eff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:54.583642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93a752-38c3-4740-8712-04b9d772eff1 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:54.583942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93a752-38c3-4740-8712-04b9d772eff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:54.584301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93a752-38c3-4740-8712-04b9d772eff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:54.595774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93a752-38c3-4740-8712-04b9d772eff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:54.597279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93a752-38c3-4740-8712-04b9d772eff1 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:54.617182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1156/2312] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:55.017339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.018820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:55.032632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd89dce-6afa-4bb1-baca-0041c449a4a6 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.043612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.044436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.045374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.046063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] PATH_INFO: `/v3/projects/c4f7007652c24550b98f5c85f9e8a0a6` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.047471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f7007652c24550b98f5c85f9e8a0a6)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.061772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.111488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:55.129170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f7007652c24550b98f5c85f9e8a0a6', 'request_id': 'req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:55.135493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f7007652c24550b98f5c85f9e8a0a6', 'request_id': 'req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:55.141493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f7007652c24550b98f5c85f9e8a0a6', 'request_id': 'req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:55.156218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] The token cache is being invalidate because project c4f7007652c24550b98f5c85f9e8a0a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:55.156506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:55.157112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-26a117b6-dbe0-47a3-8a0b-97dbaaba1737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:55.158692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1157/2313] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:55 2022] DELETE /identity/v3/projects/c4f7007652c24550b98f5c85f9e8a0a6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:55.163717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270168d-fc5f-4d9f-9ac5-fa99a5542d96 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.164023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270168d-fc5f-4d9f-9ac5-fa99a5542d96 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.164284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270168d-fc5f-4d9f-9ac5-fa99a5542d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.184661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1270168d-fc5f-4d9f-9ac5-fa99a5542d96 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:55.186021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1270168d-fc5f-4d9f-9ac5-fa99a5542d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:55.203765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:55.265728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1157/2314] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:55.268122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be35e82-4002-410a-bc89-8a1fc9dd5a00 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.268571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be35e82-4002-410a-bc89-8a1fc9dd5a00 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.268571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be35e82-4002-410a-bc89-8a1fc9dd5a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.289452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9be35e82-4002-410a-bc89-8a1fc9dd5a00 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:55.290743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9be35e82-4002-410a-bc89-8a1fc9dd5a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:55.299441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1158/2315] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:55.305072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270168d-fc5f-4d9f-9ac5-fa99a5542d96 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.317482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270168d-fc5f-4d9f-9ac5-fa99a5542d96 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.324170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.324967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.325287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.325604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.326495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:55.327303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.329180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.345849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1158/2316] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:55.350684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be35e82-4002-410a-bc89-8a1fc9dd5a00 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.361677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be35e82-4002-410a-bc89-8a1fc9dd5a00 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.369031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.369913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.370276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.370598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.371456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.375077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.415947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1159/2317] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:55 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:55.419994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.431430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04d39-3578-402d-932b-040ca428771c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.490076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.490908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.491064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.491338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.492021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.494089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.516573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1159/2318] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:55 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:55.520763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.531801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf97e2-f7a5-4afe-9b58-d33044af18d3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.536677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.537377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.537564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.537829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.538546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.540314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.551913 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1160/2319] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:55.555271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.565205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c46c3-4367-42de-99e4-66731672ae34 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.569816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8a982-d719-47d5-be57-66b9f7c6b159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.570434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8a982-d719-47d5-be57-66b9f7c6b159 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.570705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8a982-d719-47d5-be57-66b9f7c6b159 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.570946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8a982-d719-47d5-be57-66b9f7c6b159 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.571583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8a982-d719-47d5-be57-66b9f7c6b159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.573203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8a982-d719-47d5-be57-66b9f7c6b159 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.583697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1160/2320] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:55.587789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.597886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea5caf-4237-481a-98b6-5c3685feb424 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.602490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.603136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.603413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.603683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] PATH_INFO: `/v3/projects/9236e760d5d846819de1ae06a66016b8/users/7f0d2069b514458eaacb1530a11fe2b0/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.619237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9236e760d5d846819de1ae06a66016b8, user_id=7f0d2069b514458eaacb1530a11fe2b0, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.621915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.637211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1161/2321] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:55 2022] PUT /identity/v3/projects/9236e760d5d846819de1ae06a66016b8/users/7f0d2069b514458eaacb1530a11fe2b0/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:55.649901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ce1eb-63ed-4a20-91d4-820de8599f6a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.650187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ce1eb-63ed-4a20-91d4-820de8599f6a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.650457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ce1eb-63ed-4a20-91d4-820de8599f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.679975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ef4ce1eb-63ed-4a20-91d4-820de8599f6a None None] MFA Rules not processed for user `7f0d2069b514458eaacb1530a11fe2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:55.681176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ef4ce1eb-63ed-4a20-91d4-820de8599f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:55.746353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1161/2322] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:55.751063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.817076 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cb84e-a1cc-49b9-b584-595785feb24c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.823764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.824409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.824670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.824938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.825565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.827493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.856983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1162/2323] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:55 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:55.861200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ce1eb-63ed-4a20-91d4-820de8599f6a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.872937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ce1eb-63ed-4a20-91d4-820de8599f6a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.886148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:55.939304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.939960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.940260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.940510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.941137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.943140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:55.965995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1162/2324] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:55 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:55.970506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:55.981493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9a382-d905-4582-b3e3-d52988766687 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:55.986008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:55.986661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:55.986934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:55.987200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:55.988173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:55.989764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:56.000372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1163/2325] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:56.005054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:56.015167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57bd6a-4de8-4176-9822-8c12ee0ac525 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:56.020520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:56.021159 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:56.021433 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:56.021729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] PATH_INFO: `/v3/projects/6c1f4b3369fc410c8ed94e575c670700/users/fa61b7e262534757a1123bcd1a98fe6e/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:56.037533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1f4b3369fc410c8ed94e575c670700, user_id=fa61b7e262534757a1123bcd1a98fe6e, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:56.040350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:56.057794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1163/2326] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:56 2022] PUT /identity/v3/projects/6c1f4b3369fc410c8ed94e575c670700/users/fa61b7e262534757a1123bcd1a98fe6e/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:56.062820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:56.074200 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64419e-dba9-4b26-a8f9-525debe83cb2 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:56.100122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:56.137600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05fa9-5b41-4c25-963d-647a172882cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:56.138212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05fa9-5b41-4c25-963d-647a172882cf admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:56.138547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05fa9-5b41-4c25-963d-647a172882cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:56.138801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05fa9-5b41-4c25-963d-647a172882cf admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:56.139464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05fa9-5b41-4c25-963d-647a172882cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:56.141176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05fa9-5b41-4c25-963d-647a172882cf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:56.150938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1164/2327] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:56.155575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:56.169525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53c624-b0b0-469c-8bae-3083ef19eb8c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:56.175627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:56.176341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:56.176615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:56.176872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] PATH_INFO: `/v3/projects/6c1f4b3369fc410c8ed94e575c670700/users/fa61b7e262534757a1123bcd1a98fe6e/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:56.178981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1f4b3369fc410c8ed94e575c670700, user_id=fa61b7e262534757a1123bcd1a98fe6e, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:56.181658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:56.193167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1164/2328] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:56 2022] PUT /identity/v3/projects/6c1f4b3369fc410c8ed94e575c670700/users/fa61b7e262534757a1123bcd1a98fe6e/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:56.203623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:56.203916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:56.204224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:56.233603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] MFA Rules not processed for user `fa61b7e262534757a1123bcd1a98fe6e`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:56.234872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:56.313340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1165/2329] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:56.324814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:56.326857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:56.389230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b2066-9901-4f88-a311-9b68e10123b3 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:56.393399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:56.394003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:56.394298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:56.394542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:56.455486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:56.457016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:56.466272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1165/2330] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:57.611222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:57.613305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:57.653769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:57.655234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:57.685826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:57.687522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078176c-f24a-4267-bb9d-19a1135f3c84 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:57.693931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:57.694608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:57.694874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:57.695147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] PATH_INFO: `/v3/users/3bebc9b1b9c0425ea0032575a509a88f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:57.696395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bebc9b1b9c0425ea0032575a509a88f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:57.698355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:57.741245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647df15-0a8c-4e4b-b14a-05997dae50de service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:57.747239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:57.748127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:57.748295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:57.748616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] PATH_INFO: `/v3/users/4a031835644f46038310afeba5722b78` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:57.749920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a031835644f46038310afeba5722b78)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:57.752778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:57.767355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bebc9b1b9c0425ea0032575a509a88f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.768082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bebc9b1b9c0425ea0032575a509a88f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.771053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:57.781149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bebc9b1b9c0425ea0032575a509a88f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.781673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bebc9b1b9c0425ea0032575a509a88f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.799171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-e945e2ad-be00-46ac-a176-d21929e21f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bebc9b1b9c0425ea0032575a509a88f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.815629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1166/2331] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:57 2022] DELETE /identity/v3/users/3bebc9b1b9c0425ea0032575a509a88f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:57.817327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a031835644f46038310afeba5722b78'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.818158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a031835644f46038310afeba5722b78'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.818781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a031835644f46038310afeba5722b78'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.820101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb2b05-262e-4bc1-871d-99caf531eeb3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:57.820421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb2b05-262e-4bc1-871d-99caf531eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:57.820739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb2b05-262e-4bc1-871d-99caf531eeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:57.827792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a031835644f46038310afeba5722b78'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.842359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c0cb2b05-262e-4bc1-871d-99caf531eeb3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:57.843843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c0cb2b05-262e-4bc1-871d-99caf531eeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:57.852112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7472e6cf-8250-4087-8993-659b7bf871af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a031835644f46038310afeba5722b78'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:57.879539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1166/2332] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:57 2022] DELETE /identity/v3/users/4a031835644f46038310afeba5722b78 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:57.881411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:57.881699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:57.881969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:57.910665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1167/2333] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:57.911973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:57.912256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:57.912571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:57.917470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] MFA Rules not processed for user `1d22ff09fb6743c2880af0d7778fb984`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:57.918659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:57.936630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:57.938718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:58.000200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1167/2334] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:58.001056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:58.002579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:58.020398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1168/2335] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:58.033827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:58.036121 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:58.086422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:58.091618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2b133-174d-4c55-8e2f-a325bd40ae79 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:58.098506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:58.099414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:58.100214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:58.100724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:58.134105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:58.135831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:58.142296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6bd8-e6fb-49c9-a5c4-b732741e5c5e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:58.146468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1168/2336] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:58.152823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:58.153644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:58.153928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:58.154387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:58.168251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:58.169854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:58.179651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1169/2337] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:54:58.874931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:58.876524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:58.898708 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dd38a-f9fd-4b03-9406-a5a97e4cb3ca service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:58.907369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:58.908070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:58.908400 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:58.908677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] PATH_INFO: `/v3/projects/aa07ba2c096a401f9c72739e7e2ee18e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:58.910468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa07ba2c096a401f9c72739e7e2ee18e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:58.912479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:58.982036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa07ba2c096a401f9c72739e7e2ee18e', 'request_id': 'req-8549397e-56d7-4178-82a1-22f2abd81369'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:58.989468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa07ba2c096a401f9c72739e7e2ee18e', 'request_id': 'req-8549397e-56d7-4178-82a1-22f2abd81369'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.007934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa07ba2c096a401f9c72739e7e2ee18e', 'request_id': 'req-8549397e-56d7-4178-82a1-22f2abd81369'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.016404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.017925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.034886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ddc96-9bbb-4dc4-af02-9bf97c17b8d8 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.035213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] The token cache is being invalidate because project aa07ba2c096a401f9c72739e7e2ee18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:59.035660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.036705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8549397e-56d7-4178-82a1-22f2abd81369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.039553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1169/2338] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:58 2022] DELETE /identity/v3/projects/aa07ba2c096a401f9c72739e7e2ee18e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:59.046748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:59.047712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f985-2593-4759-8f86-1047b5172352 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.049029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f985-2593-4759-8f86-1047b5172352 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.049207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f985-2593-4759-8f86-1047b5172352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.073546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b555f985-2593-4759-8f86-1047b5172352 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:59.074826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b555f985-2593-4759-8f86-1047b5172352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.112645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.113177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.113673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.113989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] PATH_INFO: `/v3/projects/0f41b1696e934fd78f57519c7994e5b3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.116374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f41b1696e934fd78f57519c7994e5b3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.119030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.150210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1170/2339] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:59.158527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de2b44-4b84-42ba-85eb-27a4d02e05fe None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.158901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de2b44-4b84-42ba-85eb-27a4d02e05fe None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.159178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de2b44-4b84-42ba-85eb-27a4d02e05fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.179323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b2de2b44-4b84-42ba-85eb-27a4d02e05fe None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:59.180555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b2de2b44-4b84-42ba-85eb-27a4d02e05fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.184211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f41b1696e934fd78f57519c7994e5b3', 'request_id': 'req-3f4f686a-bbe2-474f-838a-98eda6fe37bb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.190634 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f41b1696e934fd78f57519c7994e5b3', 'request_id': 'req-3f4f686a-bbe2-474f-838a-98eda6fe37bb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.196303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f41b1696e934fd78f57519c7994e5b3', 'request_id': 'req-3f4f686a-bbe2-474f-838a-98eda6fe37bb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.211515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] The token cache is being invalidate because project 0f41b1696e934fd78f57519c7994e5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:59.211801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.212471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.213963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1170/2340] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:59 2022] DELETE /identity/v3/projects/0f41b1696e934fd78f57519c7994e5b3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:59.216681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.218073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.252331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1171/2341] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:59.257042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de2b44-4b84-42ba-85eb-27a4d02e05fe None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.269072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de2b44-4b84-42ba-85eb-27a4d02e05fe None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.274608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.275357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.275671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.275992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.276664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:59.276972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.278595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.291949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1172/2342] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:54:59.296644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.302205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:59.308760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bd9b4-b7cd-4be2-9e22-1195cbceff5c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.314986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.316106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.316490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.316859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.317622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.320477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.321978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f686a-bbe2-474f-838a-98eda6fe37bb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.327088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.327749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.328055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.328329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] PATH_INFO: `/v3/users/7f0d2069b514458eaacb1530a11fe2b0` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.329650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0d2069b514458eaacb1530a11fe2b0)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.331669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.355946 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1173/2343] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:59 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:59.360980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.398368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:59.402256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0d2069b514458eaacb1530a11fe2b0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.403327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0d2069b514458eaacb1530a11fe2b0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.416156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0d2069b514458eaacb1530a11fe2b0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.417596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0d2069b514458eaacb1530a11fe2b0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.437656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-515923b0-8fba-4f34-a964-9c30e44963c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0d2069b514458eaacb1530a11fe2b0'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:59.450360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53a79c-cd03-461d-a265-03e747de22aa admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.452439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1171/2344] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:59 2022] DELETE /identity/v3/users/7f0d2069b514458eaacb1530a11fe2b0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:59.455310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.456073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.456355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.456629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.456958 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.457242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.457359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.457545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.459166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.483086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1174/2345] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:59 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:59.487287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.489375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.502312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2a617-a156-40fd-8b32-413a61c172df admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.571528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.572215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.572485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.572745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.573402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.575059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.586135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1175/2346] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:59.590171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.601052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e82a1-6984-45a6-ad8a-9c819c5b6bc1 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.605256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:59.606305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.606497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.606907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.607181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.607451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.608124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.609732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.614419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1172/2347] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:59.621006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1176/2348] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:59.624277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.625041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.625722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.635050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7a125-e110-4722-b79e-faf470b49331 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.639850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d199a2-732c-4652-ae23-1de612b93824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.640551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d199a2-732c-4652-ae23-1de612b93824 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.640833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d199a2-732c-4652-ae23-1de612b93824 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.641107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d199a2-732c-4652-ae23-1de612b93824 admin admin] PATH_INFO: `/v3/projects/881a40342cb84608b961e6b32dceadb4/users/d8d329978baf425e954b9965ae0cb652/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.659659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d199a2-732c-4652-ae23-1de612b93824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=881a40342cb84608b961e6b32dceadb4, user_id=d8d329978baf425e954b9965ae0cb652, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.662310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d199a2-732c-4652-ae23-1de612b93824 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.678109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1177/2349] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:59 2022] PUT /identity/v3/projects/881a40342cb84608b961e6b32dceadb4/users/d8d329978baf425e954b9965ae0cb652/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:59.683176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.683463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.683736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.688282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813292c-b233-4d69-84da-7610bd5ed2a3 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.702933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:59.704202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.729882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:59.737518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.738479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.738751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.739086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.766350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1178/2350] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:59.777094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:59.778493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:59.791142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4eb36-1d48-44d5-ab05-57a91a871390 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:59.795261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:59.795933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:59.796226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:59.796684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:59.808245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.809684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.815617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:59.817232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:59.819432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1179/2351] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:54:59.826772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1173/2352] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:55:00.107595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:00.109689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:00.149102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacc66a-84ab-460f-affd-4762602a2c66 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:00.157005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:00.157581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:00.158990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:00.159339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] PATH_INFO: `/v3/projects/9236e760d5d846819de1ae06a66016b8` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:00.160758 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9236e760d5d846819de1ae06a66016b8)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:00.162905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:00.271047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9236e760d5d846819de1ae06a66016b8', 'request_id': 'req-3485efd9-e831-43bf-bff4-b0a80df8ae7d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.279904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9236e760d5d846819de1ae06a66016b8', 'request_id': 'req-3485efd9-e831-43bf-bff4-b0a80df8ae7d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.286164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9236e760d5d846819de1ae06a66016b8', 'request_id': 'req-3485efd9-e831-43bf-bff4-b0a80df8ae7d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.310699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] The token cache is being invalidate because project 9236e760d5d846819de1ae06a66016b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:55:00.311229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.313330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.316186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1180/2353] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:55:00 2022] DELETE /identity/v3/projects/9236e760d5d846819de1ae06a66016b8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:55:00.320318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:00.330175 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62736-3028-40a5-8d76-2ffe80fc2410 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:00.349184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:00.399295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:00.400097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:00.400378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:00.400654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] PATH_INFO: `/v3/users/fa61b7e262534757a1123bcd1a98fe6e` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:00.401918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa61b7e262534757a1123bcd1a98fe6e)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:00.403984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:00.463709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa61b7e262534757a1123bcd1a98fe6e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.464698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa61b7e262534757a1123bcd1a98fe6e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.473255 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa61b7e262534757a1123bcd1a98fe6e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.474394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa61b7e262534757a1123bcd1a98fe6e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.491400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6b1ca81f-d7c0-4385-a689-3e07949c62fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa61b7e262534757a1123bcd1a98fe6e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:00.511905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1174/2354] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:55:00 2022] DELETE /identity/v3/users/fa61b7e262534757a1123bcd1a98fe6e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:55:00.875089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:00.877956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:00.903391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485efd9-e831-43bf-bff4-b0a80df8ae7d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:00.990893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:00.991603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:00.991908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:00.992187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] PATH_INFO: `/v3/projects/6c1f4b3369fc410c8ed94e575c670700` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:00.993560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1f4b3369fc410c8ed94e575c670700)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:00.995647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:01.056464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:01.060415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1f4b3369fc410c8ed94e575c670700', 'request_id': 'req-fd1cba00-ad7a-40eb-a944-01ae32ad2352'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:01.069923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1f4b3369fc410c8ed94e575c670700', 'request_id': 'req-fd1cba00-ad7a-40eb-a944-01ae32ad2352'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:01.076512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1f4b3369fc410c8ed94e575c670700', 'request_id': 'req-fd1cba00-ad7a-40eb-a944-01ae32ad2352'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:01.092122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] The token cache is being invalidate because project 6c1f4b3369fc410c8ed94e575c670700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:55:01.093298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:01.094017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-fd1cba00-ad7a-40eb-a944-01ae32ad2352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:55:01.096236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1181/2355] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:55:00 2022] DELETE /identity/v3/projects/6c1f4b3369fc410c8ed94e575c670700 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:55:01.100984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e9a6d-abdf-4b31-b45a-eb7c64cc9c53 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.101253 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e9a6d-abdf-4b31-b45a-eb7c64cc9c53 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.101522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e9a6d-abdf-4b31-b45a-eb7c64cc9c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.129116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-252e9a6d-abdf-4b31-b45a-eb7c64cc9c53 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:01.133603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-252e9a6d-abdf-4b31-b45a-eb7c64cc9c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:01.213054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1175/2356] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:01.220622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ccc31-8dd0-4bd2-abb6-a8925c533416 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.221166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ccc31-8dd0-4bd2-abb6-a8925c533416 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.222168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ccc31-8dd0-4bd2-abb6-a8925c533416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.262328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2d0ccc31-8dd0-4bd2-abb6-a8925c533416 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:01.264232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2d0ccc31-8dd0-4bd2-abb6-a8925c533416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:01.280519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e9a6d-abdf-4b31-b45a-eb7c64cc9c53 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:01.280933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1182/2357] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:01.302669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e9a6d-abdf-4b31-b45a-eb7c64cc9c53 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:01.316679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:01.317332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.317690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.317888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.318549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:55:01.318910 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:01.335927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:01.358970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1176/2358] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:55:01.361875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ccc31-8dd0-4bd2-abb6-a8925c533416 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:01.377898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ccc31-8dd0-4bd2-abb6-a8925c533416 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:01.397675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:01.398306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.398578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.398847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.399478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:01.401423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:01.450297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1183/2359] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:55:01 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:55:01.454379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:01.466167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78846b65-0934-4906-bd00-753884485f76 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:01.498268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:01.570972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:01.571654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.572206 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.572512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.573154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:01.575117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:01.603899 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1177/2360] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:55:01 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:55:01.605329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:01.619459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55cf0b-b683-4121-ba09-05c5a15dcb9d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:01.628049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:01.629003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.629202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.629474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.631111 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:01.632776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:01.649782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1184/2361] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:55:01.653748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:01.670220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090333-29b6-4837-8cf5-9751fee51c90 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:01.681223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001c585-e67a-4014-9ff4-5639ab719a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:01.684361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c585-e67a-4014-9ff4-5639ab719a74 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.684361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c585-e67a-4014-9ff4-5639ab719a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.684361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c585-e67a-4014-9ff4-5639ab719a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.684361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001c585-e67a-4014-9ff4-5639ab719a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:01.685597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001c585-e67a-4014-9ff4-5639ab719a74 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:01.708258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1178/2362] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:55:01.712949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:01.726462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29688820-0ac9-4b1a-87ec-7abb977172e1 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:01.771565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:01.772462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.772756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.773008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] PATH_INFO: `/v3/projects/d66d40b700e242eebfd51f2469879a3a/users/3b636220f1dd443eacdeb3f975736dfc/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.790449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d66d40b700e242eebfd51f2469879a3a, user_id=3b636220f1dd443eacdeb3f975736dfc, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:01.793101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:01.835117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1185/2363] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:55:01 2022] PUT /identity/v3/projects/d66d40b700e242eebfd51f2469879a3a/users/3b636220f1dd443eacdeb3f975736dfc/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:55:01.840219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97a8e-86d1-4236-8f36-b0a6fd4b1a3b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:01.840539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97a8e-86d1-4236-8f36-b0a6fd4b1a3b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:01.840778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97a8e-86d1-4236-8f36-b0a6fd4b1a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:01.867720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-47e97a8e-86d1-4236-8f36-b0a6fd4b1a3b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:01.868986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-47e97a8e-86d1-4236-8f36-b0a6fd4b1a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:01.970802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1179/2364] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:01.985841 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:01.989046 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:02.070857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74887b7-3370-46db-b515-9b2eb1fcff7f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:02.075565 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:02.076237 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:02.076551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:02.076821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:02.087022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:02.088488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:02.099129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1186/2365] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:55:02.895154 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93fff8-c773-4ced-b4f6-2584485f8a6f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:02.895683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93fff8-c773-4ced-b4f6-2584485f8a6f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:02.895796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93fff8-c773-4ced-b4f6-2584485f8a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:02.932641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-fc93fff8-c773-4ced-b4f6-2584485f8a6f None None] MFA Rules not processed for user `3b636220f1dd443eacdeb3f975736dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:02.934301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fc93fff8-c773-4ced-b4f6-2584485f8a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:03.042392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1180/2366] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:03.054036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:03.055791 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:03.151319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1bb7d-9ecd-4cde-9013-ca8274e52834 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:03.163657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ccc1d-521d-4d0d-afad-b1d108171487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:03.165466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccc1d-521d-4d0d-afad-b1d108171487 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:03.166305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccc1d-521d-4d0d-afad-b1d108171487 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:03.166835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccc1d-521d-4d0d-afad-b1d108171487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:03.200106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ccc1d-521d-4d0d-afad-b1d108171487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:03.205515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ccc1d-521d-4d0d-afad-b1d108171487 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:03.230346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1187/2367] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:55:04.979701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:04.980539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:04.980812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.002211 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] MFA Rules not processed for user `3b636220f1dd443eacdeb3f975736dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:05.003676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:05.011876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1181/2368] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:05.026511 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba6ee5-6002-4edf-a0df-07873b3dab13 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:05.027050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba6ee5-6002-4edf-a0df-07873b3dab13 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:05.027390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba6ee5-6002-4edf-a0df-07873b3dab13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.053919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:05.057068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-66ba6ee5-6002-4edf-a0df-07873b3dab13 None None] MFA Rules not processed for user `d8d329978baf425e954b9965ae0cb652`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:05.058278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-66ba6ee5-6002-4edf-a0df-07873b3dab13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:05.144058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1188/2369] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:05.144909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:05.146331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:05.161804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2fe9-6d42-48e7-b3aa-9e245ec3a97a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:05.168148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:05.168773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:05.169052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:05.169465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.170140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:05.174294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:05.219774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1182/2370] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:55:05 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:55:05.221052 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba6ee5-6002-4edf-a0df-07873b3dab13 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:05.233929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba6ee5-6002-4edf-a0df-07873b3dab13 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:05.299014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:05.299641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:05.299939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:05.300208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.300887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:05.302792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:05.340327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1189/2371] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:55:05 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:55:05.344096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:05.356134 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137f5c-d9a0-4f57-b85a-40b088f1aa73 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:05.361109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:05.361725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:05.361999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:05.362264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.363252 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:05.364858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:05.366704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:05.378896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1183/2372] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:55:05.382725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:05.393830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c99be1-d952-4deb-9f09-cbb089b0919d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:05.399558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:05.399937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:05.400217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:05.400488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] PATH_INFO: `/v3/projects/7799aa2d333849f6a64bd45850efb3f5/users/1429c21cc23a440db90d172c38c92b21/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.416211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7799aa2d333849f6a64bd45850efb3f5, user_id=1429c21cc23a440db90d172c38c92b21, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:05.418965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:05.434968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1190/2373] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:55:05 2022] PUT /identity/v3/projects/7799aa2d333849f6a64bd45850efb3f5/users/1429c21cc23a440db90d172c38c92b21/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:55:05.439390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:05.452461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310fad8-ed7d-4b20-a28a-568a25b765a5 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:05.527545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d9b13-bac9-4c19-b1d1-a55ee224b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:05.528684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9b13-bac9-4c19-b1d1-a55ee224b0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:05.528684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9b13-bac9-4c19-b1d1-a55ee224b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:05.529095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9b13-bac9-4c19-b1d1-a55ee224b0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.529858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d9b13-bac9-4c19-b1d1-a55ee224b0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:05.531588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d9b13-bac9-4c19-b1d1-a55ee224b0f4 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:05.542766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1184/2374] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:55:05.546742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:05.563348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f73381-bc1c-48f2-8617-4e19dc491d0b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:05.575873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:05.577437 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:05.577955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:05.579413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] PATH_INFO: `/v3/projects/7799aa2d333849f6a64bd45850efb3f5/users/1429c21cc23a440db90d172c38c92b21/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:05.582555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7799aa2d333849f6a64bd45850efb3f5, user_id=1429c21cc23a440db90d172c38c92b21, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:05.585459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:05.601998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1191/2375] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:55:05 2022] PUT /identity/v3/projects/7799aa2d333849f6a64bd45850efb3f5/users/1429c21cc23a440db90d172c38c92b21/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:55:10.719424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0946de-d57a-4e73-935d-4f35862b6272 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:10.719996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0946de-d57a-4e73-935d-4f35862b6272 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:10.720530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0946de-d57a-4e73-935d-4f35862b6272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:10.771357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-7a0946de-d57a-4e73-935d-4f35862b6272 None None] MFA Rules not processed for user `1429c21cc23a440db90d172c38c92b21`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:10.773622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7a0946de-d57a-4e73-935d-4f35862b6272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:10.919983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1185/2376] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:10.931204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:10.933103 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:11.053633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:11.110527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68d177-9c85-4083-a2d9-61b04761c665 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:11.133309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac7f43-59d9-4c57-8b74-d0125528256a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:11.134120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7f43-59d9-4c57-8b74-d0125528256a service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:11.134516 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7f43-59d9-4c57-8b74-d0125528256a service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:11.135277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7f43-59d9-4c57-8b74-d0125528256a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:11.222080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac7f43-59d9-4c57-8b74-d0125528256a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:11.225405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac7f43-59d9-4c57-8b74-d0125528256a service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:11.240101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1192/2377] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:55:20.910606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=65 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:21.236124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=31 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:29.680608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95d3a-c618-4ccf-bfa6-f4760ae716e4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:29.681203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95d3a-c618-4ccf-bfa6-f4760ae716e4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:29.681855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95d3a-c618-4ccf-bfa6-f4760ae716e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:29.701465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b4e95d3a-c618-4ccf-bfa6-f4760ae716e4 None None] MFA Rules not processed for user `3b636220f1dd443eacdeb3f975736dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:29.702687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b4e95d3a-c618-4ccf-bfa6-f4760ae716e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:29.758161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1186/2378] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:39.751994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=25 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:53.060293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:53.060912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:53.060912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:53.080752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] MFA Rules not processed for user `d8d329978baf425e954b9965ae0cb652`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:53.082001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:53.089709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1193/2379] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:03.075426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:09.465804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f8345-7beb-49b4-8dc2-4714a8ba1bf3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:09.466499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f8345-7beb-49b4-8dc2-4714a8ba1bf3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:09.467019 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f8345-7beb-49b4-8dc2-4714a8ba1bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:09.488312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-538f8345-7beb-49b4-8dc2-4714a8ba1bf3 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:09.489829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-538f8345-7beb-49b4-8dc2-4714a8ba1bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:09.547391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1187/2380] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:09.593604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:09.595201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:09.670773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be262576-f470-4e13-a6b4-9fb12b873bd5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:09.676171 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74d411-788b-4973-92f1-458aa03fc9b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:09.676793 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d411-788b-4973-92f1-458aa03fc9b6 service swift] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:09.677074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d411-788b-4973-92f1-458aa03fc9b6 service swift] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:09.677343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d411-788b-4973-92f1-458aa03fc9b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:09.685855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74d411-788b-4973-92f1-458aa03fc9b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:09.687523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74d411-788b-4973-92f1-458aa03fc9b6 service swift] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:09.695744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1194/2381] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:56:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:56:16.910356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68aedad-9dea-41b9-aefb-375bc48704f0 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:16.910930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68aedad-9dea-41b9-aefb-375bc48704f0 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:16.910930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68aedad-9dea-41b9-aefb-375bc48704f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:16.946866 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d68aedad-9dea-41b9-aefb-375bc48704f0 None None] MFA Rules not processed for user `1e44ee032c5346ec9260740bcb700007`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:16.950032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d68aedad-9dea-41b9-aefb-375bc48704f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:17.038296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1188/2382] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:56:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:56:19.692780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=32 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:22.333473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d857a29-bc04-4cdd-8b95-725c908e2df2 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:22.334089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d857a29-bc04-4cdd-8b95-725c908e2df2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:22.334089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d857a29-bc04-4cdd-8b95-725c908e2df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:22.353820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-6d857a29-bc04-4cdd-8b95-725c908e2df2 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:22.355042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6d857a29-bc04-4cdd-8b95-725c908e2df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:22.362392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1195/2383] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:56:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:22.375630 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68aedad-9dea-41b9-aefb-375bc48704f0 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:22.388049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68aedad-9dea-41b9-aefb-375bc48704f0 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:22.393739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88c2dc-327d-4fe6-a8b3-824152c1af41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:22.394452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88c2dc-327d-4fe6-a8b3-824152c1af41 service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:22.394847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88c2dc-327d-4fe6-a8b3-824152c1af41 service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:22.395178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88c2dc-327d-4fe6-a8b3-824152c1af41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:22.406577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88c2dc-327d-4fe6-a8b3-824152c1af41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:22.407226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88c2dc-327d-4fe6-a8b3-824152c1af41 service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:22.415597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1189/2384] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:56:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:56:28.225681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f27f6d-ad26-4983-94e4-a0d1824b5e6c None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.226806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f27f6d-ad26-4983-94e4-a0d1824b5e6c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.227271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f27f6d-ad26-4983-94e4-a0d1824b5e6c None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.229314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1196/2385] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:56:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:56:28.239111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac12de-43a9-4a24-81b5-9516197ed212 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.239758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac12de-43a9-4a24-81b5-9516197ed212 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.240411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac12de-43a9-4a24-81b5-9516197ed212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.262995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d8ac12de-43a9-4a24-81b5-9516197ed212 None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:28.264960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d8ac12de-43a9-4a24-81b5-9516197ed212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.278705 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1190/2386] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:56:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:28.293337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f27f6d-ad26-4983-94e4-a0d1824b5e6c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:28.297664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-41f27f6d-ad26-4983-94e4-a0d1824b5e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.406506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f27f6d-ad26-4983-94e4-a0d1824b5e6c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:28.412852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45db89d-b6f2-49e9-b7fe-971dd59c2b1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:28.413777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db89d-b6f2-49e9-b7fe-971dd59c2b1b service glance] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.414359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db89d-b6f2-49e9-b7fe-971dd59c2b1b service glance] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.414808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45db89d-b6f2-49e9-b7fe-971dd59c2b1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.430602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45db89d-b6f2-49e9-b7fe-971dd59c2b1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:28.433353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45db89d-b6f2-49e9-b7fe-971dd59c2b1b service glance] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:28.461591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1197/2387] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:28.543172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bcbce-4387-423b-806d-dce7bf1c4b6d None None] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.544044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bcbce-4387-423b-806d-dce7bf1c4b6d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.544515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bcbce-4387-423b-806d-dce7bf1c4b6d None None] PATH_INFO: `/` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.547378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1191/2388] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:56:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:56:28.553462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221361dd-ec83-47ed-9a5e-a199b1c33e4f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.553740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221361dd-ec83-47ed-9a5e-a199b1c33e4f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.554037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221361dd-ec83-47ed-9a5e-a199b1c33e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.576001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-221361dd-ec83-47ed-9a5e-a199b1c33e4f None None] MFA Rules not processed for user `349ee49c297749468e09c4aa3dc9c218`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:28.577246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-221361dd-ec83-47ed-9a5e-a199b1c33e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.638625 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1198/2389] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:56:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:28.652382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bcbce-4387-423b-806d-dce7bf1c4b6d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:28.664170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bcbce-4387-423b-806d-dce7bf1c4b6d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:28.668021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e4159-f237-4e39-87c4-bd64e6bbf93c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:28.668695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e4159-f237-4e39-87c4-bd64e6bbf93c service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.669201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e4159-f237-4e39-87c4-bd64e6bbf93c service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.669669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e4159-f237-4e39-87c4-bd64e6bbf93c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.678896 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e4159-f237-4e39-87c4-bd64e6bbf93c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:28.681571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e4159-f237-4e39-87c4-bd64e6bbf93c service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:28.692381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1192/2390] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:28.821832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5348ff6-e7dc-46d4-a665-0df3cbf0a268 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.822850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5348ff6-e7dc-46d4-a665-0df3cbf0a268 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.823191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5348ff6-e7dc-46d4-a665-0df3cbf0a268 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.825157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1199/2391] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:56:28.846966 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d795f7-68ad-4507-8300-537bbe66d27c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.847338 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d795f7-68ad-4507-8300-537bbe66d27c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.847698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d795f7-68ad-4507-8300-537bbe66d27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.877934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c6d795f7-68ad-4507-8300-537bbe66d27c None None] MFA Rules not processed for user `bf149f246c1644aa85c4c162a2e07984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:28.880344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c6d795f7-68ad-4507-8300-537bbe66d27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.935364 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1193/2392] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:56:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:28.957358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5348ff6-e7dc-46d4-a665-0df3cbf0a268 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:28.959399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a5348ff6-e7dc-46d4-a665-0df3cbf0a268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.982680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5348ff6-e7dc-46d4-a665-0df3cbf0a268 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:28.986879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:28.987512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.987788 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.988120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:29.023356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:29.024862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:29.048731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1200/2393] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:56:31.888602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d795f7-68ad-4507-8300-537bbe66d27c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:31.893650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=69 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:31.902261 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d795f7-68ad-4507-8300-537bbe66d27c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:31.906543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:31.907170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:31.907449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:31.907723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:31.909488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:31.922949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:31.924494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:31.935661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1194/2394] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:33.184604 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:33.197124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd0bd-c15e-4e8c-bd59-4091b0a0fa37 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:33.202235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6749dc-c90f-4ad2-96ff-58e2bd4a865a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:33.203327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6749dc-c90f-4ad2-96ff-58e2bd4a865a service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:33.203786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6749dc-c90f-4ad2-96ff-58e2bd4a865a service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:33.204252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6749dc-c90f-4ad2-96ff-58e2bd4a865a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:33.206537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9c6749dc-c90f-4ad2-96ff-58e2bd4a865a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:33.245839 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6749dc-c90f-4ad2-96ff-58e2bd4a865a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:33.249602 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6749dc-c90f-4ad2-96ff-58e2bd4a865a service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:33.304270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1201/2395] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:33.768723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:33.778590 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb558d-f7dc-4f23-ac2f-2bd34e2b959c service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:33.783299 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:33.783936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:33.784393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:33.784664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:33.795175 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:33.796627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:33.808763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1195/2396] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:36.885252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e3b0d-db29-4b6b-986b-9f98acb26de9 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:36.886287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e3b0d-db29-4b6b-986b-9f98acb26de9 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:36.886789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e3b0d-db29-4b6b-986b-9f98acb26de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:36.929984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-b42e3b0d-db29-4b6b-986b-9f98acb26de9 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:36.931690 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b42e3b0d-db29-4b6b-986b-9f98acb26de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:36.940613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1202/2397] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:56:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:36.955990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:36.971077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2c551-2ebe-42d1-8829-308f4d4b785d service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:36.981385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:36.982563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:36.983032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:36.983833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:37.001537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:37.005027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:37.016734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1196/2398] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:56:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:56:40.759967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0906f2-3001-4ea8-8426-95ea00560494 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:40.760487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0906f2-3001-4ea8-8426-95ea00560494 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:40.760578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0906f2-3001-4ea8-8426-95ea00560494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:40.780186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1f0906f2-3001-4ea8-8426-95ea00560494 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:40.781417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1f0906f2-3001-4ea8-8426-95ea00560494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:40.792050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1203/2399] 10.222.0.44 () {58 vars in 967 bytes} [Tue Oct 4 08:56:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:40.807280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:40.818277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c637f-7984-4f2a-bb60-b8db2b782bf0 service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:40.823923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5777f07-5899-4244-8358-7f5bb2ad290e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:40.824422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5777f07-5899-4244-8358-7f5bb2ad290e service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:40.824895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5777f07-5899-4244-8358-7f5bb2ad290e service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:40.825181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5777f07-5899-4244-8358-7f5bb2ad290e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:40.833989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5777f07-5899-4244-8358-7f5bb2ad290e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:40.835500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5777f07-5899-4244-8358-7f5bb2ad290e service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:40.843585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1197/2400] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:56:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:56:43.212842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ba68-7371-4d11-a9c9-1890a7971d66 None None] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:43.213460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ba68-7371-4d11-a9c9-1890a7971d66 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:43.213720 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ba68-7371-4d11-a9c9-1890a7971d66 None None] PATH_INFO: `/` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:43.216011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1204/2401] 10.222.0.44 () {54 vars in 921 bytes} [Tue Oct 4 08:56:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:56:43.226511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:43.227085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:43.227085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:43.248202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] MFA Rules not processed for user `9cee970f95754740b2f19e64f4653c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:43.249783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:43.293112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ba68-7371-4d11-a9c9-1890a7971d66 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:43.348270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1198/2402] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Oct 4 08:56:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:43.349696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:43.351150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:43.365555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d598e2e-f0d2-4cd9-8b08-72c5a0131314 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:43.371911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:43.372576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:43.372851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:43.373119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:43.381843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:43.381996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ba68-7371-4d11-a9c9-1890a7971d66 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:43.383416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:43.386317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:43.386937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:43.387212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:43.387480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] PATH_INFO: `/v3/users/3b636220f1dd443eacdeb3f975736dfc` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:43.388647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b636220f1dd443eacdeb3f975736dfc)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:43.390539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:43.393104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1199/2403] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:43.396816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:43.444166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b636220f1dd443eacdeb3f975736dfc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.444900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b636220f1dd443eacdeb3f975736dfc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.445559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b636220f1dd443eacdeb3f975736dfc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.452611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b636220f1dd443eacdeb3f975736dfc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.467906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a621ac3e-21fd-4117-a006-b0174d63efb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b636220f1dd443eacdeb3f975736dfc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.488502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1205/2404] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:56:43 2022] DELETE /identity/v3/users/3b636220f1dd443eacdeb3f975736dfc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:56:43.679968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:43.692167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddf7f8-078c-4495-bf37-b0de0b7dee1f service placement] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:43.749603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:43.750146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:43.750421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:43.750695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] PATH_INFO: `/v3/projects/d66d40b700e242eebfd51f2469879a3a` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:43.751796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d66d40b700e242eebfd51f2469879a3a)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:43.753754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:43.807865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:43.824073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd66d40b700e242eebfd51f2469879a3a', 'request_id': 'req-a5320dd2-7be9-46ec-b06e-706c674d9311'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.830945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd66d40b700e242eebfd51f2469879a3a', 'request_id': 'req-a5320dd2-7be9-46ec-b06e-706c674d9311'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.836978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd66d40b700e242eebfd51f2469879a3a', 'request_id': 'req-a5320dd2-7be9-46ec-b06e-706c674d9311'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.851291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] The token cache is being invalidate because project d66d40b700e242eebfd51f2469879a3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:56:43.851687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.852337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a5320dd2-7be9-46ec-b06e-706c674d9311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:43.857148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1200/2405] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:56:43 2022] DELETE /identity/v3/projects/d66d40b700e242eebfd51f2469879a3a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:56:43.858849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78452c-c9e5-44b4-a7e7-11c75431e050 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:43.859133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78452c-c9e5-44b4-a7e7-11c75431e050 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:43.859406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78452c-c9e5-44b4-a7e7-11c75431e050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:43.879538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-2b78452c-c9e5-44b4-a7e7-11c75431e050 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:43.880816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2b78452c-c9e5-44b4-a7e7-11c75431e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:43.950726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1206/2406] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:43.960027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bf4a0-2284-4503-89a0-98ef732dc151 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:43.960369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bf4a0-2284-4503-89a0-98ef732dc151 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:43.960611 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bf4a0-2284-4503-89a0-98ef732dc151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:43.981217 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-818bf4a0-2284-4503-89a0-98ef732dc151 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:43.982445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-818bf4a0-2284-4503-89a0-98ef732dc151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:43.992359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1201/2407] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:43.996479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78452c-c9e5-44b4-a7e7-11c75431e050 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:44.008756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78452c-c9e5-44b4-a7e7-11c75431e050 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:44.020267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:44.020914 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.021201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.021477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.022182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:56:44.022487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:44.024239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:44.036962 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1207/2408] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:56:44.041574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bf4a0-2284-4503-89a0-98ef732dc151 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:44.051390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bf4a0-2284-4503-89a0-98ef732dc151 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:44.056437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:44.057122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.057486 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.057824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.058521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:44.060739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:44.090199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1202/2409] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:56:44 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:56:44.095372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:44.106938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664451-b106-45de-aa98-1b8630eaa370 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:44.166513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:44.167197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.167475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.167746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.168485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:44.170416 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:44.199846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1208/2410] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:56:44 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:56:44.204511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:44.216215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59262ef5-a47d-4c8a-8a02-654b6ed450e6 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:44.221492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:44.222323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.222789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.223215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.224081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:44.226016 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:44.237395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1203/2411] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:56:44.243399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:44.248721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:44.255494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c9e03-f127-41ce-92d6-33b06136429a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:44.262213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8b06d-3b51-42af-9026-e90ea439e2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:44.262837 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8b06d-3b51-42af-9026-e90ea439e2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.263116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8b06d-3b51-42af-9026-e90ea439e2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.263423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8b06d-3b51-42af-9026-e90ea439e2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.264220 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8b06d-3b51-42af-9026-e90ea439e2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:44.265809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8b06d-3b51-42af-9026-e90ea439e2c7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:44.277709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1209/2412] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:56:44.282157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:44.292245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d66887-422c-4f5d-a54e-527f56fcf267 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:44.296738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:44.297381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.297655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.297920 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] PATH_INFO: `/v3/projects/0aa17d0bd0134273b27f58ed3be0b320/users/bca0489c7dd347e8931184835accd348/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.312042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa17d0bd0134273b27f58ed3be0b320, user_id=bca0489c7dd347e8931184835accd348, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:44.314662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:44.329489 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1204/2413] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:56:44 2022] PUT /identity/v3/projects/0aa17d0bd0134273b27f58ed3be0b320/users/bca0489c7dd347e8931184835accd348/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:56:44.340621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.340912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.341295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.370289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] MFA Rules not processed for user `bca0489c7dd347e8931184835accd348`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:44.371511 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:44.441398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1210/2414] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:56:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:44.454715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:44.457099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:44.514610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab0ef-0415-4093-93ba-5c0ee9f19feb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:44.518196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:44.518796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:44.519070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:44.519335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:44.528008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:44.529434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:44.538598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1205/2415] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:46.132218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:46.133768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:46.209296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b01cbf-57b6-494e-8893-6d5b7112dbfd None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:46.214733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:46.215338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:46.215611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:46.215879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:46.217643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:46.283945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:46.285632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:46.295770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1211/2416] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:54.534216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:56.290816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:57.450278 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:57.451760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:57.525115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:57.527451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:57.564980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770aa74-d701-4aec-aa90-e9418440f68f service glance] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:57.568693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9a790-776e-4b91-a279-35262dc098d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:57.569313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9a790-776e-4b91-a279-35262dc098d0 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:57.569593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9a790-776e-4b91-a279-35262dc098d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:57.569861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9a790-776e-4b91-a279-35262dc098d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:57.571605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-34b9a790-776e-4b91-a279-35262dc098d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:57.582436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9a790-776e-4b91-a279-35262dc098d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:57.584201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9a790-776e-4b91-a279-35262dc098d0 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:57.593930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1206/2417] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:57.680879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef00f2-c8e6-44cd-b0e5-2578a54cd68f service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:57.688218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:57.689004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:57.689218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:57.689562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:57.708574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:57.710025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:57.719294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1212/2418] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:07.590765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=26 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:07.715219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=26 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:24.426056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e55a24-4772-4b50-945f-dcf03469e4a2 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:24.426566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e55a24-4772-4b50-945f-dcf03469e4a2 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:24.426660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e55a24-4772-4b50-945f-dcf03469e4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:24.447511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-96e55a24-4772-4b50-945f-dcf03469e4a2 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:24.448736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-96e55a24-4772-4b50-945f-dcf03469e4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:24.517842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1207/2419] 10.222.0.44 () {58 vars in 967 bytes} [Tue Oct 4 08:57:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:24.544094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:24.545902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:24.620035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10666e-56c3-4836-b053-a9ae73f4a0f9 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:24.625058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:24.625688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:24.625975 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:24.626238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:24.634517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:24.636130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:24.644367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1213/2420] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:57:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:57:34.509276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=25 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:34.641330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=32 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:46.623993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e55a24-4772-4b50-945f-dcf03469e4a2 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:46.637608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e55a24-4772-4b50-945f-dcf03469e4a2 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:46.643823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69335f77-b41f-41bc-b126-aeb0c6613403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:46.644454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69335f77-b41f-41bc-b126-aeb0c6613403 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:46.644748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69335f77-b41f-41bc-b126-aeb0c6613403 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:46.645015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69335f77-b41f-41bc-b126-aeb0c6613403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:46.647126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-69335f77-b41f-41bc-b126-aeb0c6613403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:46.660789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69335f77-b41f-41bc-b126-aeb0c6613403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:46.662426 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69335f77-b41f-41bc-b126-aeb0c6613403 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:46.673566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1208/2421] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:56.503701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:56.570817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86db2f9-3d72-4e01-9b55-1ec376bff7c1 service swift] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:56.575480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:56.576130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:56.576407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:56.576677 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] PATH_INFO: `/v3/users/bca0489c7dd347e8931184835accd348` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:56.577801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca0489c7dd347e8931184835accd348)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:56.579718 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:56.626374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca0489c7dd347e8931184835accd348'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:56.627013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca0489c7dd347e8931184835accd348'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:56.627608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca0489c7dd347e8931184835accd348'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:56.634457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca0489c7dd347e8931184835accd348'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:56.646934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca0489c7dd347e8931184835accd348'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:56.661597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1214/2422] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:56 2022] DELETE /identity/v3/users/bca0489c7dd347e8931184835accd348 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:56.667149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:56.667677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:56.667776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:56.671967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=6 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:56.692380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:56.693718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:56.753827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1209/2423] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:57:56.763812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:56.765315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:56.823493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30fefd-8d71-4317-969f-d22a1a329d11 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:56.828148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3773d-66d6-453e-9f84-4358d3251fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:56.828746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3773d-66d6-453e-9f84-4358d3251fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:56.829016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3773d-66d6-453e-9f84-4358d3251fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:56.829279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3773d-66d6-453e-9f84-4358d3251fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:56.838803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3773d-66d6-453e-9f84-4358d3251fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:56.840252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3773d-66d6-453e-9f84-4358d3251fc0 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:56.849655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1215/2424] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:57.122749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.124433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:57.148251 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8344f5-efd5-4a15-a098-2109e0e45529 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.155181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.157418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.157418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.157418 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] PATH_INFO: `/v3/projects/0aa17d0bd0134273b27f58ed3be0b320` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.158303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa17d0bd0134273b27f58ed3be0b320)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.160339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.210707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa17d0bd0134273b27f58ed3be0b320', 'request_id': 'req-92ce1656-8bb1-464f-81e2-aa63139c3f30'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:57.219386 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa17d0bd0134273b27f58ed3be0b320', 'request_id': 'req-92ce1656-8bb1-464f-81e2-aa63139c3f30'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:57.225988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa17d0bd0134273b27f58ed3be0b320', 'request_id': 'req-92ce1656-8bb1-464f-81e2-aa63139c3f30'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:57.240292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] The token cache is being invalidate because project 0aa17d0bd0134273b27f58ed3be0b320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:57.240579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:57.241210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-92ce1656-8bb1-464f-81e2-aa63139c3f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:57.242713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1210/2425] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:57 2022] DELETE /identity/v3/projects/0aa17d0bd0134273b27f58ed3be0b320 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:57.247916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1b138-a5de-4e6e-9f6d-1260f3bff3fd None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.248285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1b138-a5de-4e6e-9f6d-1260f3bff3fd None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.248620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1b138-a5de-4e6e-9f6d-1260f3bff3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.269321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5ef1b138-a5de-4e6e-9f6d-1260f3bff3fd None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:57.270540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5ef1b138-a5de-4e6e-9f6d-1260f3bff3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:57.292266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:57.343069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1216/2426] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:57.352679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376ec4-d3fa-4173-b793-507633f9e0ed None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.352968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376ec4-d3fa-4173-b793-507633f9e0ed None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.353259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376ec4-d3fa-4173-b793-507633f9e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.377078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-fd376ec4-d3fa-4173-b793-507633f9e0ed None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:57.378321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fd376ec4-d3fa-4173-b793-507633f9e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:57.387801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1211/2427] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:57.392061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1b138-a5de-4e6e-9f6d-1260f3bff3fd None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.403184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1b138-a5de-4e6e-9f6d-1260f3bff3fd None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.408163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.408951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.409266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.409633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.410540 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:57.410936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.412672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.431535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1217/2428] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:57.433085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376ec4-d3fa-4173-b793-507633f9e0ed None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.447764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376ec4-d3fa-4173-b793-507633f9e0ed None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.452864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.453494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.453769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.454036 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.454653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.456632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.487139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1212/2429] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:57 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:57.492307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.502480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3df0c-631c-4880-bebc-7ccbf659cb03 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.561226 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.561894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.562107 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.562378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.563060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.565007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.588176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1218/2430] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:57 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:57:57.592375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.607071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8193-e9d5-4271-b42d-427a222d1275 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.613368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.614021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.614298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.614568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.615565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.617209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.628103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1213/2431] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:57.632378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.643039 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a4b0-8fdf-49e2-a177-bcbdb99de8ec admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.648136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7bf59-23cf-416d-a385-02fbe2773e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.648738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7bf59-23cf-416d-a385-02fbe2773e25 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.649047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7bf59-23cf-416d-a385-02fbe2773e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.649331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7bf59-23cf-416d-a385-02fbe2773e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.649980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7bf59-23cf-416d-a385-02fbe2773e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.651577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7bf59-23cf-416d-a385-02fbe2773e25 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.662239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1219/2432] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:57.666636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.676853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e11bb8-0b01-4c45-afdf-ce2011ca3c8f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.682264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.682941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.683223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.683495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] PATH_INFO: `/v3/projects/1fa3abe3bbbf48cd9a72e82743df055b/users/5bce779b68e048c7a4da02989505642d/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.698970 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa3abe3bbbf48cd9a72e82743df055b, user_id=5bce779b68e048c7a4da02989505642d, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.701652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.715761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1214/2433] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:57 2022] PUT /identity/v3/projects/1fa3abe3bbbf48cd9a72e82743df055b/users/5bce779b68e048c7a4da02989505642d/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:57.726272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006165ca-de17-41ea-94f2-c2bd21e8bad4 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.726552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006165ca-de17-41ea-94f2-c2bd21e8bad4 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.726818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006165ca-de17-41ea-94f2-c2bd21e8bad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.762645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-006165ca-de17-41ea-94f2-c2bd21e8bad4 None None] MFA Rules not processed for user `5bce779b68e048c7a4da02989505642d`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:57.764154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-006165ca-de17-41ea-94f2-c2bd21e8bad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:57.830771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1220/2434] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:57:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:57.842126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:57.843993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:57.872921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:57.911393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46039d-7cdf-4880-8e37-e6f216d81b26 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:57.915459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aa5db-80d0-4a42-9364-d3708fdef470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:57.916155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aa5db-80d0-4a42-9364-d3708fdef470 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:57.916427 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aa5db-80d0-4a42-9364-d3708fdef470 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:57.917351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aa5db-80d0-4a42-9364-d3708fdef470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:57.926148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aa5db-80d0-4a42-9364-d3708fdef470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:57.927568 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aa5db-80d0-4a42-9364-d3708fdef470 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:57.936492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1215/2435] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:07.827256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:07.932638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=17 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:14.447085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006165ca-de17-41ea-94f2-c2bd21e8bad4 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:14.512858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006165ca-de17-41ea-94f2-c2bd21e8bad4 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:14.517786 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:14.518424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:14.518689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:14.518958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] PATH_INFO: `/v3/users/5bce779b68e048c7a4da02989505642d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:14.520154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bce779b68e048c7a4da02989505642d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:14.522069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:14.572754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bce779b68e048c7a4da02989505642d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:14.573678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bce779b68e048c7a4da02989505642d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:14.574308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bce779b68e048c7a4da02989505642d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:14.582055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bce779b68e048c7a4da02989505642d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:14.595040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bce779b68e048c7a4da02989505642d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:14.610023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1221/2436] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:58:14 2022] DELETE /identity/v3/users/5bce779b68e048c7a4da02989505642d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:14.614520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:14.614982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:14.615113 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:14.635007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:14.636824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:14.701388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1216/2437] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:14.711531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:14.713200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:14.771414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa042c0-1296-430c-84bc-0f3e3a36b409 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:14.775179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:14.775783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:14.776085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:14.776353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:14.785568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:14.786993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:14.797231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1222/2438] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:14.975434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:14.977196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:14.989115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929a2b-2d27-4056-aa7c-4ae47872290f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:14.994519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:14.995157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:14.995502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:14.995773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] PATH_INFO: `/v3/users/1e44ee032c5346ec9260740bcb700007` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:14.997029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e44ee032c5346ec9260740bcb700007)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:14.999034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.047334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e44ee032c5346ec9260740bcb700007'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.048139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e44ee032c5346ec9260740bcb700007'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.054935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e44ee032c5346ec9260740bcb700007'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.055559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e44ee032c5346ec9260740bcb700007'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.067719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a4bea614-88cd-4654-8d9d-4fdf4df09d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e44ee032c5346ec9260740bcb700007'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.083838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1217/2439] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 08:58:14 2022] DELETE /identity/v3/users/1e44ee032c5346ec9260740bcb700007 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:15.090298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.092157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:15.106600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806455d-0792-4a3d-92e5-f2978f76f19f service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.128482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:15.169326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.169961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.170234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.170504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] PATH_INFO: `/v3/projects/1fa3abe3bbbf48cd9a72e82743df055b` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.171646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa3abe3bbbf48cd9a72e82743df055b)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.173579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.221334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa3abe3bbbf48cd9a72e82743df055b', 'request_id': 'req-01fe1d46-de35-45ae-8db5-dbdee14ac31c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.227378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa3abe3bbbf48cd9a72e82743df055b', 'request_id': 'req-01fe1d46-de35-45ae-8db5-dbdee14ac31c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.233028 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa3abe3bbbf48cd9a72e82743df055b', 'request_id': 'req-01fe1d46-de35-45ae-8db5-dbdee14ac31c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.247478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] The token cache is being invalidate because project 1fa3abe3bbbf48cd9a72e82743df055b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:58:15.247767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.248456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.250058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1223/2440] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:58:15 2022] DELETE /identity/v3/projects/1fa3abe3bbbf48cd9a72e82743df055b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:15.255180 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e71b-5898-4283-86b8-aed1a16a4838 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.255461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e71b-5898-4283-86b8-aed1a16a4838 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.255728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e71b-5898-4283-86b8-aed1a16a4838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.261831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.263307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:15.278010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0924e71b-5898-4283-86b8-aed1a16a4838 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:15.279214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0924e71b-5898-4283-86b8-aed1a16a4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:15.339370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1218/2441] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:15.348323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1d46-de35-45ae-8db5-dbdee14ac31c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.348622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78ed1-b519-403f-a545-f2dfe8956f64 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.348908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78ed1-b519-403f-a545-f2dfe8956f64 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.349183 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78ed1-b519-403f-a545-f2dfe8956f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.352943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.353572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.353845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.354113 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] PATH_INFO: `/v3/projects/67b474ba7f7a465caa2ed14f891ac951` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.355199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b474ba7f7a465caa2ed14f891ac951)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.357110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.370257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d2d78ed1-b519-403f-a545-f2dfe8956f64 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:15.371464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d2d78ed1-b519-403f-a545-f2dfe8956f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:15.379446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1219/2442] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:58:15.383573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78ed1-b519-403f-a545-f2dfe8956f64 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.392794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78ed1-b519-403f-a545-f2dfe8956f64 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.396676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:15.405458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b474ba7f7a465caa2ed14f891ac951', 'request_id': 'req-7f20fe62-7c03-42c4-9a46-d84c78698912'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.411751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b474ba7f7a465caa2ed14f891ac951', 'request_id': 'req-7f20fe62-7c03-42c4-9a46-d84c78698912'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.413008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:15.419319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b474ba7f7a465caa2ed14f891ac951', 'request_id': 'req-7f20fe62-7c03-42c4-9a46-d84c78698912'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.435824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] The token cache is being invalidate because project 67b474ba7f7a465caa2ed14f891ac951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:58:15.436122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.437158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:15.439041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1224/2443] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 08:58:15 2022] DELETE /identity/v3/projects/67b474ba7f7a465caa2ed14f891ac951 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:58:15.454924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.455561 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.455839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.456193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.456864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:58:15.457167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.458820 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.472918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1220/2444] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 08:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:58:15.477886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.479418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:15.494040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20fe62-7c03-42c4-9a46-d84c78698912 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.499102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.499711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.500019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.500309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.500951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.502995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.536002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1225/2445] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:58:15 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:58:15.540517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.551782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3c9ab-eef3-4643-96d2-a3f8e292a2bf admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.625352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.626301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.626752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.627156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.628178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.631303 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.663465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1221/2446] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:58:15 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:58:15.669329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.681217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0fa56-120a-4cb1-8be6-a589f5f76ba5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.686983 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.687596 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.687859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.688153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.688802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.690436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.701234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1226/2447] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:58:15.705743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.720322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffc77c-d402-45d8-b46c-1b753c850e76 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.725487 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aabc5a-f795-4998-8fba-9ce0d0d8846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.726081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aabc5a-f795-4998-8fba-9ce0d0d8846b admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.726357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aabc5a-f795-4998-8fba-9ce0d0d8846b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.726625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aabc5a-f795-4998-8fba-9ce0d0d8846b admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.727258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aabc5a-f795-4998-8fba-9ce0d0d8846b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.728876 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aabc5a-f795-4998-8fba-9ce0d0d8846b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.739772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1222/2448] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:58:15.744078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.754455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c722797-a2c4-48dc-ba10-0885691f828c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.759551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.760225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.760495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.760761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] PATH_INFO: `/v3/projects/62a1577879c64dc6aac3d5fd4d7ad637/users/3fb4f49a5be0475889369e13c3bc1481/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.775664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a1577879c64dc6aac3d5fd4d7ad637, user_id=3fb4f49a5be0475889369e13c3bc1481, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.778310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.794048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1227/2449] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:58:15 2022] PUT /identity/v3/projects/62a1577879c64dc6aac3d5fd4d7ad637/users/3fb4f49a5be0475889369e13c3bc1481/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:15.799047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306c49c-c9a9-43b1-a62e-f14569d7f81f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.799325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306c49c-c9a9-43b1-a62e-f14569d7f81f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.799593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306c49c-c9a9-43b1-a62e-f14569d7f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.818463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-f306c49c-c9a9-43b1-a62e-f14569d7f81f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:15.819658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f306c49c-c9a9-43b1-a62e-f14569d7f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:15.883572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1223/2450] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:15.894011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:15.896818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:15.971539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc42814-a0f7-4aff-99f3-5299251d707a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:15.975710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:15.976339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:15.976614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:15.976878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:15.986309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:15.987713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:15.997490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1228/2451] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:19.900201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2b43d-fe24-4998-8850-03ef2039f817 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:19.900658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2b43d-fe24-4998-8850-03ef2039f817 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:19.900658 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2b43d-fe24-4998-8850-03ef2039f817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:19.936733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-dcb2b43d-fe24-4998-8850-03ef2039f817 None None] MFA Rules not processed for user `3fb4f49a5be0475889369e13c3bc1481`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:19.938052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-dcb2b43d-fe24-4998-8850-03ef2039f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:19.946713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:20.011178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1224/2452] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:20.014044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:20.024208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6a121-0638-41af-a891-7bb95853f079 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:20.028808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:20.029421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:20.029702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:20.029976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:20.030626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:20.032647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:20.064576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1229/2453] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 08:58:20 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:58:20.068291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2b43d-fe24-4998-8850-03ef2039f817 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:20.079601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2b43d-fe24-4998-8850-03ef2039f817 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:20.140114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:20.141050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:20.141324 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:20.141588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:20.142230 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:20.144164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:20.167929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1225/2454] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 08:58:20 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:58:20.171955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:20.182776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55978857-644e-4d9e-b090-730b953c605f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:20.187247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:20.187906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:20.188223 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:20.188496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:20.189148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:20.190745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:20.202204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1230/2455] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:58:20.206755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:20.217656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21960b-f3b9-4b2d-8b59-03c23bff7a82 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:20.222904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:20.223609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:20.223916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:20.224226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] PATH_INFO: `/v3/projects/0f35bffe6f7e4d899340dc4f9781b8c3/users/6fed61b47090479ba4b158a5033645d6/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:20.239872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f35bffe6f7e4d899340dc4f9781b8c3, user_id=6fed61b47090479ba4b158a5033645d6, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:20.242909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:20.257757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1226/2456] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:58:20 2022] PUT /identity/v3/projects/0f35bffe6f7e4d899340dc4f9781b8c3/users/6fed61b47090479ba4b158a5033645d6/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:20.261869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:20.279065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd669ab-2fc6-4a2e-aaee-37b03dffc0b4 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:20.340956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:20.349308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47eb0a-5cab-4cf8-986a-95dae4a78bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:20.350584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47eb0a-5cab-4cf8-986a-95dae4a78bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:20.351279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47eb0a-5cab-4cf8-986a-95dae4a78bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:20.351538 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47eb0a-5cab-4cf8-986a-95dae4a78bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:20.352354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47eb0a-5cab-4cf8-986a-95dae4a78bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:20.354587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47eb0a-5cab-4cf8-986a-95dae4a78bb1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:20.365780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1231/2457] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 08:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:58:20.369988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:20.380822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4d214-e239-4e80-97c8-299ac2d9284f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:20.385848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:20.386523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:20.386795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:20.387059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] PATH_INFO: `/v3/projects/0f35bffe6f7e4d899340dc4f9781b8c3/users/6fed61b47090479ba4b158a5033645d6/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:20.389055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f35bffe6f7e4d899340dc4f9781b8c3, user_id=6fed61b47090479ba4b158a5033645d6, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:20.391750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:20.409967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1227/2458] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 08:58:20 2022] PUT /identity/v3/projects/0f35bffe6f7e4d899340dc4f9781b8c3/users/6fed61b47090479ba4b158a5033645d6/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:24.759902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404db578-e68c-4d9a-ab99-87412e19b94f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:24.760490 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404db578-e68c-4d9a-ab99-87412e19b94f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:24.760612 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404db578-e68c-4d9a-ab99-87412e19b94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:24.789164 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-404db578-e68c-4d9a-ab99-87412e19b94f None None] MFA Rules not processed for user `6fed61b47090479ba4b158a5033645d6`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:24.790393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-404db578-e68c-4d9a-ab99-87412e19b94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:24.863600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1232/2459] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:24.876720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:24.878627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:24.951632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664ffaf-a9c7-4e73-a7a3-568d6d86abe5 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:24.956739 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:24.957365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:24.957638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:24.957898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:24.974932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:25.023365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:25.025006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:25.034473 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1228/2460] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:58:25.200739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:25.201024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:25.201325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:25.221922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:25.223158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:25.282989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1233/2461] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:58:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:25.348768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:25.350233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:25.423018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abaf11-8a54-45e1-8612-d5b0fddc388d service glance] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:25.429033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03cd99-47c0-49e4-8ea1-1736e2498bab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:25.429657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cd99-47c0-49e4-8ea1-1736e2498bab service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:25.429936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cd99-47c0-49e4-8ea1-1736e2498bab service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:25.430362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cd99-47c0-49e4-8ea1-1736e2498bab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:25.439144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03cd99-47c0-49e4-8ea1-1736e2498bab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:25.440802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03cd99-47c0-49e4-8ea1-1736e2498bab service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:25.448578 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1229/2462] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:58:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:58:26.425253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:26.427493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:26.552887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68b05f-2253-4209-ad93-7b185821fb0a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:26.559124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:26.559754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:26.560671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:26.560949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:26.562821 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:26.587230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:26.641239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:26.642573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:26.655151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1234/2463] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:34.303610 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:34.304243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:34.304243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:34.323533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] MFA Rules not processed for user `3fb4f49a5be0475889369e13c3bc1481`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:34.324875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:34.331907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1230/2464] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:34.594187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:34.649257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8477d-5c4e-48dc-bf73-16eade863711 service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:34.653202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519c8d3-195a-41fd-b443-2342733752d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:34.653898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519c8d3-195a-41fd-b443-2342733752d7 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:34.654357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519c8d3-195a-41fd-b443-2342733752d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:34.654687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519c8d3-195a-41fd-b443-2342733752d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:34.657055 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5519c8d3-195a-41fd-b443-2342733752d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:34.668424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519c8d3-195a-41fd-b443-2342733752d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:34.668814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519c8d3-195a-41fd-b443-2342733752d7 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:34.677722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1235/2465] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:35.008214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:35.009949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:35.106455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fe4ad-3b54-4fcc-8cd3-e749bffb7040 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:35.112098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:35.112098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:35.112371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:35.112646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:35.114452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:35.126928 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:35.128870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:35.138950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1231/2466] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:36.896415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:36.896908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:36.897040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:36.917065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:36.918184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:36.926704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1236/2467] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 08:58:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:58:36.939799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:36.952575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79dd9b-b707-4033-8cc9-4ef1de94486b service placement] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:36.958232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:36.958861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:36.959171 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:36.959524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:36.970028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:36.970028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:36.978451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1232/2468] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 08:58:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:58:46.911216 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=48 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:46.975006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=93 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:51.816460 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:51.818847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:51.839295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47864cb-1753-4969-9fd3-775cf70d2939 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:51.846068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:51.846890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:51.847505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:51.847680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:51.850086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:51.862842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:51.864714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:51.876459 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1237/2469] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:58.517955 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:58.530395 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031cd2c-5fdc-4a35-8f0d-f846cfed77cf service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:58.536840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98533359-76a8-4869-8d7c-abc52b2f8409 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:58.537443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533359-76a8-4869-8d7c-abc52b2f8409 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:58.537723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533359-76a8-4869-8d7c-abc52b2f8409 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:58.537984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98533359-76a8-4869-8d7c-abc52b2f8409 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:58.539976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-98533359-76a8-4869-8d7c-abc52b2f8409 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:58.600321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98533359-76a8-4869-8d7c-abc52b2f8409 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:58.601893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98533359-76a8-4869-8d7c-abc52b2f8409 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:58.610829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1233/2470] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:59.549194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:59.562384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd61bb-67c9-4fe8-850a-65d229fe2d1b service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:59.571062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f7595-f668-4a90-8c2f-eddfabb0bb18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:59.571683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f7595-f668-4a90-8c2f-eddfabb0bb18 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:59.571995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f7595-f668-4a90-8c2f-eddfabb0bb18 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:59.572249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f7595-f668-4a90-8c2f-eddfabb0bb18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:59.574337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b42f7595-f668-4a90-8c2f-eddfabb0bb18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:59.588977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f7595-f668-4a90-8c2f-eddfabb0bb18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:59.590958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f7595-f668-4a90-8c2f-eddfabb0bb18 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:59.602717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1238/2471] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:59.964411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f171a-243c-4334-be28-b6a5e2ebd846 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:59.964981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f171a-243c-4334-be28-b6a5e2ebd846 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:59.964981 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f171a-243c-4334-be28-b6a5e2ebd846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:59.986081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0f6f171a-243c-4334-be28-b6a5e2ebd846 None None] MFA Rules not processed for user `1d22ff09fb6743c2880af0d7778fb984`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:59.987307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0f6f171a-243c-4334-be28-b6a5e2ebd846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:59.997140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1234/2472] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:58:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:59:08.267475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:08.268149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:08.268149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:08.288003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] MFA Rules not processed for user `68288f4e2f57454fbb06eff0dd72cca3`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:59:08.289240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:08.350385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1239/2473] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 08:59:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:59:08.955111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f171a-243c-4334-be28-b6a5e2ebd846 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:08.965459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f171a-243c-4334-be28-b6a5e2ebd846 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:08.974693 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:08.975316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:08.975591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:08.975860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:08.977709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:08.988012 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:08.989437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:08.997932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1235/2474] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:59:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:59:14.499283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:14.501033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:14.513881 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3108a5b-f0ba-4ed9-bbdc-c63ef0b154f1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:14.517842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dc93f-e10c-4235-b4a8-2634218ef2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:14.518442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dc93f-e10c-4235-b4a8-2634218ef2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:14.518731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dc93f-e10c-4235-b4a8-2634218ef2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:14.519168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dc93f-e10c-4235-b4a8-2634218ef2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:14.529481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136dc93f-e10c-4235-b4a8-2634218ef2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:14.530913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136dc93f-e10c-4235-b4a8-2634218ef2b1 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:14.541024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1240/2475] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 08:59:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:59:18.994249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=37 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:59:24.536357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=43 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:05.242575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:05.244205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:05.337544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93edfc-8a82-4ee9-ac6a-71bdc4888cd9 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:05.343067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:05.343684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:05.343995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:05.344268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:05.346199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:05.359090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:05.360748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:05.371403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1236/2476] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:06.214030 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c7df0-fc2a-4338-b99c-a70c417e2522 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:06.214552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c7df0-fc2a-4338-b99c-a70c417e2522 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:06.214552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c7df0-fc2a-4338-b99c-a70c417e2522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:06.234176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f09c7df0-fc2a-4338-b99c-a70c417e2522 None None] MFA Rules not processed for user `d8d329978baf425e954b9965ae0cb652`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:06.235314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f09c7df0-fc2a-4338-b99c-a70c417e2522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:06.313555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1241/2477] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:10.519277 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:10.550615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918c48a-d7a0-4fd4-b421-81dba6a8455f service placement] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:10.557643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:10.558238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:10.558515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:10.558783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:10.560786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:10.571834 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:10.573429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:10.582650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1237/2478] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:16.306247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=25 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:20.578342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=50 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:28.726306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c7df0-fc2a-4338-b99c-a70c417e2522 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:28.748035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c7df0-fc2a-4338-b99c-a70c417e2522 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:28.752097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509879fe-864e-43ce-912d-4615016772be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:28.752695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509879fe-864e-43ce-912d-4615016772be service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:28.752965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509879fe-864e-43ce-912d-4615016772be service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:28.753233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509879fe-864e-43ce-912d-4615016772be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:28.755142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-509879fe-864e-43ce-912d-4615016772be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:28.843834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509879fe-864e-43ce-912d-4615016772be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:28.845647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509879fe-864e-43ce-912d-4615016772be service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:28.874672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1242/2479] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:32.012640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:32.014092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:32.031312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9f2e2-024b-4f6a-a714-7926da3d5542 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:32.035772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:32.036459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:32.036729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:32.036992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] PATH_INFO: `/v3/users/1d22ff09fb6743c2880af0d7778fb984` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:32.038114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d22ff09fb6743c2880af0d7778fb984)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:32.040049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:32.089374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d22ff09fb6743c2880af0d7778fb984'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.090022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d22ff09fb6743c2880af0d7778fb984'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.097101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d22ff09fb6743c2880af0d7778fb984'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.097723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d22ff09fb6743c2880af0d7778fb984'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.111492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-953bc6c6-78ed-44b4-8af3-d1e12ffa1056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d22ff09fb6743c2880af0d7778fb984'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.127403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1238/2480] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:32 2022] DELETE /identity/v3/users/1d22ff09fb6743c2880af0d7778fb984 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:32.390259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509879fe-864e-43ce-912d-4615016772be service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:32.402407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509879fe-864e-43ce-912d-4615016772be service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:32.461857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:32.462483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:32.462795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:32.463041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] PATH_INFO: `/v3/projects/ba66b134261b494b964e9f050449a0fe` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:32.464270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba66b134261b494b964e9f050449a0fe)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:32.466175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:32.515764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba66b134261b494b964e9f050449a0fe', 'request_id': 'req-6e633aa3-e2e9-4443-812d-0235fc93289f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.522510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba66b134261b494b964e9f050449a0fe', 'request_id': 'req-6e633aa3-e2e9-4443-812d-0235fc93289f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.527012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:32.529655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba66b134261b494b964e9f050449a0fe', 'request_id': 'req-6e633aa3-e2e9-4443-812d-0235fc93289f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.547003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] The token cache is being invalidate because project ba66b134261b494b964e9f050449a0fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:32.547292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.547977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6e633aa3-e2e9-4443-812d-0235fc93289f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:32.549477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1243/2481] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:32 2022] DELETE /identity/v3/projects/ba66b134261b494b964e9f050449a0fe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:32.556794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b50e72-ac3e-44a7-a3de-48788f71d8c1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:32.557218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b50e72-ac3e-44a7-a3de-48788f71d8c1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:32.557624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b50e72-ac3e-44a7-a3de-48788f71d8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:32.629029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-48b50e72-ac3e-44a7-a3de-48788f71d8c1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:32.630853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-48b50e72-ac3e-44a7-a3de-48788f71d8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:32.693376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1239/2482] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:32.702363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac86d9b-d41c-4097-9882-4f27f44b36e5 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:32.702645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac86d9b-d41c-4097-9882-4f27f44b36e5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:32.702919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac86d9b-d41c-4097-9882-4f27f44b36e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:32.724732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-3ac86d9b-d41c-4097-9882-4f27f44b36e5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:32.725939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3ac86d9b-d41c-4097-9882-4f27f44b36e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:32.735514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1244/2483] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:00:32.740131 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b50e72-ac3e-44a7-a3de-48788f71d8c1 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:32.750806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b50e72-ac3e-44a7-a3de-48788f71d8c1 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:32.756337 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:32.756932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:32.757188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:32.757447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:32.758088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:00:32.758385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:32.760051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:32.772170 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1240/2484] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:00:32.776648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac86d9b-d41c-4097-9882-4f27f44b36e5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:32.787988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac86d9b-d41c-4097-9882-4f27f44b36e5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:32.793765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:32.794380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:32.794656 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:32.794922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:32.795558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:32.797525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:32.831157 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1245/2485] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:32 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:32.836926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:32.847446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7fe31-ed71-4289-8218-d1e99a508bbf admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:32.879039 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:32.929967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:32.930916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:32.931351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:32.931821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:32.933119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:32.936306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:32.970335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1241/2486] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:32 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:32.974117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:32.989057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09957-553b-4c22-bdd6-8ca9852cad5d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:32.999166 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:32.999765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.000074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.000336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.000984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.002597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.013239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1246/2487] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:33.017991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.031350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1009e8-059d-4e5d-b20b-9d423569777f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.036933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da557484-efe1-45f8-8cad-8b62d2bb0dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.037536 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da557484-efe1-45f8-8cad-8b62d2bb0dba admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.037795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da557484-efe1-45f8-8cad-8b62d2bb0dba admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.038064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da557484-efe1-45f8-8cad-8b62d2bb0dba admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.038702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da557484-efe1-45f8-8cad-8b62d2bb0dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.040326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da557484-efe1-45f8-8cad-8b62d2bb0dba admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.050845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1242/2488] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:33.055320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.065646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd017ab-dac6-4846-bb5f-38a623134d6f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.070669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.071338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.071580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.071852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] PATH_INFO: `/v3/projects/0ba75dcad9c14b6491784c112ceada20/users/c45393ce73e94d1597da6406e9071977/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.086471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba75dcad9c14b6491784c112ceada20, user_id=c45393ce73e94d1597da6406e9071977, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.089136 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.103876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1247/2489] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:33 2022] PUT /identity/v3/projects/0ba75dcad9c14b6491784c112ceada20/users/c45393ce73e94d1597da6406e9071977/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:33.114451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834359d-b6dc-4493-81ac-d7baccf61a12 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.114748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834359d-b6dc-4493-81ac-d7baccf61a12 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.115017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834359d-b6dc-4493-81ac-d7baccf61a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.145244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-d834359d-b6dc-4493-81ac-d7baccf61a12 None None] MFA Rules not processed for user `c45393ce73e94d1597da6406e9071977`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:33.146474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d834359d-b6dc-4493-81ac-d7baccf61a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:33.213137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1243/2490] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:33.217986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.281875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac776f-5fb1-44e2-8371-f71c90ac6213 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.287684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.288344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.288620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.288888 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.289559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.291482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.325475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1248/2491] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:33 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:33.330010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834359d-b6dc-4493-81ac-d7baccf61a12 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.341627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834359d-b6dc-4493-81ac-d7baccf61a12 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.403709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.404382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.404657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.404906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.405535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.407503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.431804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1244/2492] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:33 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:00:33.435778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.445967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a14b8-39ba-4c32-8d68-36df2143eded admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.450736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.451334 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.451607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.451871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.452530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.454109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.465295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1249/2493] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:00:33.469602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.480074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a103c-3019-4b95-afdd-1a085a7fd542 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.485666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.486321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.486585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.486853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] PATH_INFO: `/v3/projects/3fe839d3c66841dc9467820a0a8600a2/users/fcd013e4773142a2b62eb41b5ac558d9/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.502663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe839d3c66841dc9467820a0a8600a2, user_id=fcd013e4773142a2b62eb41b5ac558d9, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.505695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.519684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1245/2494] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:33 2022] PUT /identity/v3/projects/3fe839d3c66841dc9467820a0a8600a2/users/fcd013e4773142a2b62eb41b5ac558d9/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:33.523780 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.534128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ba921-ed65-441b-a3f0-a3a26360766c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.585926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:33.593060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655deb-6e7a-4ab2-ba3e-0b77cbac50d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.593723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655deb-6e7a-4ab2-ba3e-0b77cbac50d6 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.594245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655deb-6e7a-4ab2-ba3e-0b77cbac50d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.594675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655deb-6e7a-4ab2-ba3e-0b77cbac50d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.595409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655deb-6e7a-4ab2-ba3e-0b77cbac50d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.597481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655deb-6e7a-4ab2-ba3e-0b77cbac50d6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.607641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1250/2495] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:00:33.612310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.618539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:33.625623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc579-4d46-4493-9784-7d8b91c31d76 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.630636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.631320 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.631597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.631870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] PATH_INFO: `/v3/projects/3fe839d3c66841dc9467820a0a8600a2/users/fcd013e4773142a2b62eb41b5ac558d9/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.633802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe839d3c66841dc9467820a0a8600a2, user_id=fcd013e4773142a2b62eb41b5ac558d9, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.636514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.647244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1246/2496] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:33 2022] PUT /identity/v3/projects/3fe839d3c66841dc9467820a0a8600a2/users/fcd013e4773142a2b62eb41b5ac558d9/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:33.659651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab2333-0d3c-4b4d-93e8-c0c028650926 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.660089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab2333-0d3c-4b4d-93e8-c0c028650926 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.660363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab2333-0d3c-4b4d-93e8-c0c028650926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.690017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-80ab2333-0d3c-4b4d-93e8-c0c028650926 None None] MFA Rules not processed for user `fcd013e4773142a2b62eb41b5ac558d9`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:33.691229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-80ab2333-0d3c-4b4d-93e8-c0c028650926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:33.774079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1251/2497] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:33.785880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.787579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:33.872067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62a495-166c-4377-a930-86d23c6312e0 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.875044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ecf92-0768-4e7d-97d6-097e9e88f8d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.875959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ecf92-0768-4e7d-97d6-097e9e88f8d7 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.876239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ecf92-0768-4e7d-97d6-097e9e88f8d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.876507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ecf92-0768-4e7d-97d6-097e9e88f8d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:33.885925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ecf92-0768-4e7d-97d6-097e9e88f8d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:33.887471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ecf92-0768-4e7d-97d6-097e9e88f8d7 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:33.899472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1247/2498] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:33.979633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab2333-0d3c-4b4d-93e8-c0c028650926 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:33.989818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab2333-0d3c-4b4d-93e8-c0c028650926 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:33.994045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:33.994652 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:33.994934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:33.995211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:34.053789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:34.055256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:34.065117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1252/2499] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:39.920619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29343d-a243-44d7-9e7f-cdfe1b62fa31 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:39.921131 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29343d-a243-44d7-9e7f-cdfe1b62fa31 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:39.921131 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29343d-a243-44d7-9e7f-cdfe1b62fa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:39.940862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-df29343d-a243-44d7-9e7f-cdfe1b62fa31 None None] MFA Rules not processed for user `e71984f02fcd4deba5e1430ec85b8705`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:39.942055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-df29343d-a243-44d7-9e7f-cdfe1b62fa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:40.000511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1248/2500] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:44.060638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=65 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:45.126272 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:45.193100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fec8a4-4142-4159-adf7-80ae029f180a service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:45.197920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:45.198590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:45.198869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:45.199210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] PATH_INFO: `/v3/users/c45393ce73e94d1597da6406e9071977` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:45.200343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45393ce73e94d1597da6406e9071977)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:45.202259 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:45.252222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45393ce73e94d1597da6406e9071977'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.252878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45393ce73e94d1597da6406e9071977'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.253474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45393ce73e94d1597da6406e9071977'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.260785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45393ce73e94d1597da6406e9071977'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.272585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45393ce73e94d1597da6406e9071977'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.292002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1253/2501] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:45 2022] DELETE /identity/v3/users/c45393ce73e94d1597da6406e9071977 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:45.296892 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:45.297374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:45.297374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:45.317102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:45.318327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:45.378737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1249/2502] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:00:45.388585 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:45.390076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:45.463844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5f81f-4d05-4243-ba4e-05d87f996bc6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:45.467933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:45.468655 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:45.468824 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:45.469109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:45.478485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:45.479913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:45.490592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1254/2503] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 09:00:45.862707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:45.864257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:45.883697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87521-387c-4ba2-840c-f9dc15761ea7 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:45.889447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:45.890055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:45.890440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:45.890678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] PATH_INFO: `/v3/projects/0ba75dcad9c14b6491784c112ceada20` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:45.891823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba75dcad9c14b6491784c112ceada20)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:45.893761 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:45.905678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:45.943034 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba75dcad9c14b6491784c112ceada20', 'request_id': 'req-56bee04e-3007-499b-b35a-55e6e748fa4a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.949281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba75dcad9c14b6491784c112ceada20', 'request_id': 'req-56bee04e-3007-499b-b35a-55e6e748fa4a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.955220 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba75dcad9c14b6491784c112ceada20', 'request_id': 'req-56bee04e-3007-499b-b35a-55e6e748fa4a'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.970803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] The token cache is being invalidate because project 0ba75dcad9c14b6491784c112ceada20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:45.971088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.971707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:45.973264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1250/2504] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:45 2022] DELETE /identity/v3/projects/0ba75dcad9c14b6491784c112ceada20 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:45.977500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:45.987341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d40b2f-407b-481f-ab46-12e132015631 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:45.995993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:46.049776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:46.050402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.050696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.050960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] PATH_INFO: `/v3/users/fcd013e4773142a2b62eb41b5ac558d9` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.052119 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd013e4773142a2b62eb41b5ac558d9)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:46.054031 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:46.111564 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd013e4773142a2b62eb41b5ac558d9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.112312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd013e4773142a2b62eb41b5ac558d9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.112943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd013e4773142a2b62eb41b5ac558d9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.120246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd013e4773142a2b62eb41b5ac558d9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.132251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2e39cc0c-52b0-4722-9107-3740d66c9693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd013e4773142a2b62eb41b5ac558d9'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.146196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1255/2505] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:45 2022] DELETE /identity/v3/users/fcd013e4773142a2b62eb41b5ac558d9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:46.435185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:46.437500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:46.454124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bee04e-3007-499b-b35a-55e6e748fa4a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:46.517410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:46.518057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.518333 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.518603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] PATH_INFO: `/v3/projects/3fe839d3c66841dc9467820a0a8600a2` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.520075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe839d3c66841dc9467820a0a8600a2)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:46.522050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:46.570313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe839d3c66841dc9467820a0a8600a2', 'request_id': 'req-0cca9583-2502-481b-960c-10de97d692c6'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.577003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe839d3c66841dc9467820a0a8600a2', 'request_id': 'req-0cca9583-2502-481b-960c-10de97d692c6'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.582557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe839d3c66841dc9467820a0a8600a2', 'request_id': 'req-0cca9583-2502-481b-960c-10de97d692c6'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.596582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] The token cache is being invalidate because project 3fe839d3c66841dc9467820a0a8600a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:46.596875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.597466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-0cca9583-2502-481b-960c-10de97d692c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:46.598990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1251/2506] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:46 2022] DELETE /identity/v3/projects/3fe839d3c66841dc9467820a0a8600a2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:46.604835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30c46-af5c-4e5a-b4a2-ca706584780b None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.605118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30c46-af5c-4e5a-b4a2-ca706584780b None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.605494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30c46-af5c-4e5a-b4a2-ca706584780b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.625573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-8bc30c46-af5c-4e5a-b4a2-ca706584780b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:46.626755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8bc30c46-af5c-4e5a-b4a2-ca706584780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:46.690952 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1256/2507] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:46.698968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0a240-25c6-4c9b-91fe-8f55f91bb7e6 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.699318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0a240-25c6-4c9b-91fe-8f55f91bb7e6 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.699591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0a240-25c6-4c9b-91fe-8f55f91bb7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.722791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-6fd0a240-25c6-4c9b-91fe-8f55f91bb7e6 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:46.724092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6fd0a240-25c6-4c9b-91fe-8f55f91bb7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:46.737370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1252/2508] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:46.738214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30c46-af5c-4e5a-b4a2-ca706584780b None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:46.749904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30c46-af5c-4e5a-b4a2-ca706584780b None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:46.756377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:46.757044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.757341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.757607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.758273 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:00:46.758574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:46.760266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:46.774751 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1257/2509] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 09:00:46.779393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0a240-25c6-4c9b-91fe-8f55f91bb7e6 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:46.790213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0a240-25c6-4c9b-91fe-8f55f91bb7e6 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:46.795573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:46.796250 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.796532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.796804 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.797472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:46.799521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:46.835744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1253/2510] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:46 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:00:46.837053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:46.848412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b748d-0112-4f1d-9e19-057c5d34fe81 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:46.863412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:46.915994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:46.916683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.917178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.917453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.919923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:46.920711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:46.944066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1258/2511] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:46 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:46.948608 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:46.959269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:46.965450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a7e6d-faa5-4fe5-825b-35fd33d16b86 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:46.971591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:46.972390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:46.972806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:46.973895 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:46.975140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:46.977020 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:46.991929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1254/2512] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:46.994709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.005096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d1c76-3825-46f5-8e98-5088ac00daf1 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.011008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48053-a3e0-4956-b6c1-b1096d3ee329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.011876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48053-a3e0-4956-b6c1-b1096d3ee329 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.012361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48053-a3e0-4956-b6c1-b1096d3ee329 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.012667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48053-a3e0-4956-b6c1-b1096d3ee329 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.013542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e48053-a3e0-4956-b6c1-b1096d3ee329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.015160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e48053-a3e0-4956-b6c1-b1096d3ee329 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.028048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1259/2513] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:47.032389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.045992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ee11b-3902-48d3-8b14-2f89030bfa8a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.055930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.056614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.056888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.057152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] PATH_INFO: `/v3/projects/001e4dc470744fb4b761e15d6927f8d9/users/4e049f1308914eeb9f66afd623f56503/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.072971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] RBAC: Authorizing `identity:create_grant(project_id=001e4dc470744fb4b761e15d6927f8d9, user_id=4e049f1308914eeb9f66afd623f56503, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.075704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.093589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1255/2514] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:47 2022] PUT /identity/v3/projects/001e4dc470744fb4b761e15d6927f8d9/users/4e049f1308914eeb9f66afd623f56503/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:47.104875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c402f-6185-4307-9897-23d97000b537 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.105155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c402f-6185-4307-9897-23d97000b537 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.105453 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c402f-6185-4307-9897-23d97000b537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.152933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-7e8c402f-6185-4307-9897-23d97000b537 None None] MFA Rules not processed for user `4e049f1308914eeb9f66afd623f56503`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:47.154954 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7e8c402f-6185-4307-9897-23d97000b537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:47.226611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1260/2515] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:47.231209 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.293758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f58d1c-f6c8-4f85-9f65-217efb5f566a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.298450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.299061 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.299342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.299615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.300300 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.302218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.330417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1256/2516] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:47 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:47.334748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c402f-6185-4307-9897-23d97000b537 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.345439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c402f-6185-4307-9897-23d97000b537 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.403010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.403616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.403919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.404200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.404840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.406734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.429196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1261/2517] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:47 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:47.433439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.444637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a4eb2-990d-4ced-87e1-9966aa0a7b24 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.450478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.451108 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.451384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.451649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.452328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.453972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.465930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1257/2518] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:47.470244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.480717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbc7e1-9e9e-4122-a09f-5a0be1f86745 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.485750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.486412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.486694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.486968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] PATH_INFO: `/v3/projects/7a1f7489575e475784e83191976a3682/users/0eb7369734b846398584152db771b36f/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.499867 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:47.503283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1f7489575e475784e83191976a3682, user_id=0eb7369734b846398584152db771b36f, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.506350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.521147 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1262/2519] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:47 2022] PUT /identity/v3/projects/7a1f7489575e475784e83191976a3682/users/0eb7369734b846398584152db771b36f/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:00:47.525309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.536455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0592e-b4bf-4abd-8d04-ef3eaef95999 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.598361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494d54c-4d31-4a33-97cb-d7a02ed0926d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.598984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d54c-4d31-4a33-97cb-d7a02ed0926d admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.599261 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d54c-4d31-4a33-97cb-d7a02ed0926d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.599529 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494d54c-4d31-4a33-97cb-d7a02ed0926d admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.600210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494d54c-4d31-4a33-97cb-d7a02ed0926d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.601821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494d54c-4d31-4a33-97cb-d7a02ed0926d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.612465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1258/2520] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:00:47.616887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.628675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376b034-442c-48c7-b2fd-f45ee0875508 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.634285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.635065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.635407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.635743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] PATH_INFO: `/v3/projects/7a1f7489575e475784e83191976a3682/users/0eb7369734b846398584152db771b36f/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.638068 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1f7489575e475784e83191976a3682, user_id=0eb7369734b846398584152db771b36f, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.640833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.652580 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1263/2521] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:47 2022] PUT /identity/v3/projects/7a1f7489575e475784e83191976a3682/users/0eb7369734b846398584152db771b36f/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:47.663857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ef33-f5c5-4dbd-85eb-f336d46ab519 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.664193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ef33-f5c5-4dbd-85eb-f336d46ab519 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.664461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ef33-f5c5-4dbd-85eb-f336d46ab519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.694973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-e738ef33-f5c5-4dbd-85eb-f336d46ab519 None None] MFA Rules not processed for user `0eb7369734b846398584152db771b36f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:47.696213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e738ef33-f5c5-4dbd-85eb-f336d46ab519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:47.737758 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:47.804259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1259/2522] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:47.817187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:47.819026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:47.877315 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da09241-3142-4dab-9801-1a6fb8fa3c8b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:47.880980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe460d-e941-4dfe-9c24-d9d4a60c0025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:47.881582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe460d-e941-4dfe-9c24-d9d4a60c0025 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:47.881858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe460d-e941-4dfe-9c24-d9d4a60c0025 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:47.882128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe460d-e941-4dfe-9c24-d9d4a60c0025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:47.890985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe460d-e941-4dfe-9c24-d9d4a60c0025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:47.892445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe460d-e941-4dfe-9c24-d9d4a60c0025 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:47.901818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1264/2523] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:47.976550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738ef33-f5c5-4dbd-85eb-f336d46ab519 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:48.044009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738ef33-f5c5-4dbd-85eb-f336d46ab519 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:48.054574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:48.055224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:48.055496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:48.055767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] PATH_INFO: `/v3/users/4e049f1308914eeb9f66afd623f56503` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:48.056983 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e049f1308914eeb9f66afd623f56503)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:48.059142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:48.111286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e049f1308914eeb9f66afd623f56503'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.111957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e049f1308914eeb9f66afd623f56503'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.119842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e049f1308914eeb9f66afd623f56503'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.120509 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e049f1308914eeb9f66afd623f56503'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.134187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e049f1308914eeb9f66afd623f56503'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.149109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1260/2524] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:47 2022] DELETE /identity/v3/users/4e049f1308914eeb9f66afd623f56503 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:48.153729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:48.154211 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:48.154384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:48.174262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:48.175471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:48.245333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1265/2525] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:48.255999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:48.257497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:48.317125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:48.334685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99454b-8401-42cf-9b90-1009f2ca8bea admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:48.338461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:48.339066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:48.339336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:48.339599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:48.348701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:48.350130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:48.362207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1261/2526] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:48.671661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:48.673191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:48.686268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e6016-85da-4554-84c3-b1f95899fd31 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:48.692135 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:48.692774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:48.693572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:48.693848 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] PATH_INFO: `/v3/projects/001e4dc470744fb4b761e15d6927f8d9` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:48.695221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=001e4dc470744fb4b761e15d6927f8d9)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:48.697264 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:48.745890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001e4dc470744fb4b761e15d6927f8d9', 'request_id': 'req-42c26598-6403-4e60-9704-8516aaaf24ca'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.752050 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001e4dc470744fb4b761e15d6927f8d9', 'request_id': 'req-42c26598-6403-4e60-9704-8516aaaf24ca'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.758007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001e4dc470744fb4b761e15d6927f8d9', 'request_id': 'req-42c26598-6403-4e60-9704-8516aaaf24ca'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.774873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] The token cache is being invalidate because project 001e4dc470744fb4b761e15d6927f8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:48.775128 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.775873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.777884 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1266/2527] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:48 2022] DELETE /identity/v3/projects/001e4dc470744fb4b761e15d6927f8d9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:48.787389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:48.800865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cb8cb-e5ad-4a6e-b9d3-9775a6c408e0 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:48.861384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:48.862056 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:48.862334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:48.862727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] PATH_INFO: `/v3/users/0eb7369734b846398584152db771b36f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:48.863949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb7369734b846398584152db771b36f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:48.865909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:48.916669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb7369734b846398584152db771b36f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.917309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb7369734b846398584152db771b36f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.924940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb7369734b846398584152db771b36f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.925653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb7369734b846398584152db771b36f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.939312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c633d124-6623-4eb5-85ad-47855ac28f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb7369734b846398584152db771b36f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:48.960132 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1262/2528] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:48 2022] DELETE /identity/v3/users/0eb7369734b846398584152db771b36f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:49.247702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:49.251731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:49.263579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:49.267989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c26598-6403-4e60-9704-8516aaaf24ca admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:49.337528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:49.338168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.338441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.338709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] PATH_INFO: `/v3/projects/7a1f7489575e475784e83191976a3682` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.340087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1f7489575e475784e83191976a3682)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:49.342007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:49.392069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1f7489575e475784e83191976a3682', 'request_id': 'req-32f82f38-e650-4de0-aac7-e68af73b7be2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:49.398292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1f7489575e475784e83191976a3682', 'request_id': 'req-32f82f38-e650-4de0-aac7-e68af73b7be2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:49.404126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1f7489575e475784e83191976a3682', 'request_id': 'req-32f82f38-e650-4de0-aac7-e68af73b7be2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:49.421006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] The token cache is being invalidate because project 7a1f7489575e475784e83191976a3682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:49.421300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:49.422104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-32f82f38-e650-4de0-aac7-e68af73b7be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:49.423639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1267/2529] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:49 2022] DELETE /identity/v3/projects/7a1f7489575e475784e83191976a3682 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:49.429932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e7e77-d11e-4926-a276-9b0aae4ffe22 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.430225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e7e77-d11e-4926-a276-9b0aae4ffe22 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.430499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e7e77-d11e-4926-a276-9b0aae4ffe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.450519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-be7e7e77-d11e-4926-a276-9b0aae4ffe22 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:49.451774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-be7e7e77-d11e-4926-a276-9b0aae4ffe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:49.516458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1263/2530] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:49.524629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c4f6a-31c6-406f-9fc8-bab77f14a2fe None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.524904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c4f6a-31c6-406f-9fc8-bab77f14a2fe None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.525181 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c4f6a-31c6-406f-9fc8-bab77f14a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.544903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-425c4f6a-31c6-406f-9fc8-bab77f14a2fe None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:49.546100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-425c4f6a-31c6-406f-9fc8-bab77f14a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:49.555726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1268/2531] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:49.560078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e7e77-d11e-4926-a276-9b0aae4ffe22 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:49.566178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:49.570565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e7e77-d11e-4926-a276-9b0aae4ffe22 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:49.577254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:49.577879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.578168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.578451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.579543 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:00:49.579872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:49.581531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:49.593592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1264/2532] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:00:49.598187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c4f6a-31c6-406f-9fc8-bab77f14a2fe None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:49.607390 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c4f6a-31c6-406f-9fc8-bab77f14a2fe None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:49.612445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:49.613009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.613280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.613547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.614230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:49.616121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:49.645519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1269/2533] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:49 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:49.649813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:49.659795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ee40d-bc83-4a22-bb95-25d9f57b6255 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:49.718844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:49.719434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.719773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.720082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.720748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:49.722740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:49.746646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1265/2534] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:49 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:49.751008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:49.762021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ae3d3-e06c-4003-a047-1da8192e30bd admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:49.766857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:49.767462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.767741 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.768054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.768727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:49.770308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:49.780378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1270/2535] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:49.784546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:49.794745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690c844-d6a9-4851-936c-9afa0d6a57cc admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:49.799480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89da9fd-5ea3-4003-bf31-fc9380d07c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:49.800104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da9fd-5ea3-4003-bf31-fc9380d07c3b admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.800385 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da9fd-5ea3-4003-bf31-fc9380d07c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.800653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da9fd-5ea3-4003-bf31-fc9380d07c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.801294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89da9fd-5ea3-4003-bf31-fc9380d07c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:49.802887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89da9fd-5ea3-4003-bf31-fc9380d07c3b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:49.812668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1266/2536] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:49.816768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:49.829755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea6c3c-250e-4987-82b9-8864efb00182 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:49.835045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:49.835699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.836008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.836284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] PATH_INFO: `/v3/projects/7196c12c54b6476c816970eea2aa53d0/users/742eb4bbe39346b991cf316cbb329e23/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.851295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7196c12c54b6476c816970eea2aa53d0, user_id=742eb4bbe39346b991cf316cbb329e23, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:49.853951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:49.869073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1271/2537] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:49 2022] PUT /identity/v3/projects/7196c12c54b6476c816970eea2aa53d0/users/742eb4bbe39346b991cf316cbb329e23/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:49.879647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22aed-1773-40e4-9cd6-aedd4ff6a4a1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:49.879933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22aed-1773-40e4-9cd6-aedd4ff6a4a1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:49.880247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22aed-1773-40e4-9cd6-aedd4ff6a4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:49.908452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-33b22aed-1773-40e4-9cd6-aedd4ff6a4a1 None None] MFA Rules not processed for user `742eb4bbe39346b991cf316cbb329e23`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:49.909683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-33b22aed-1773-40e4-9cd6-aedd4ff6a4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:49.974865 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1267/2538] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:49.979756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.015056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:50.047646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d82f9-a0ba-42d6-ab4c-f04e66d695c3 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.052647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.053270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.053570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.053832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.054465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.056415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.087057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1272/2539] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:49 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:50.091506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b22aed-1773-40e4-9cd6-aedd4ff6a4a1 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.102823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b22aed-1773-40e4-9cd6-aedd4ff6a4a1 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.162452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.163096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.163379 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.163648 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.164312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.166242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.174241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:50.189519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1268/2540] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:50 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:50.193682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.204495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d4cf7-fd00-4b8b-8ed1-433d4e5558a5 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.209425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.210023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.210338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.210603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.211242 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.213183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.227787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1273/2541] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:50.232186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.243586 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3138f-857a-4e92-8f95-95739182ea69 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.249309 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccd472-5aa4-453e-966b-401f0e0acb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.249914 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd472-5aa4-453e-966b-401f0e0acb72 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.250207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd472-5aa4-453e-966b-401f0e0acb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.250467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccd472-5aa4-453e-966b-401f0e0acb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.251141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccd472-5aa4-453e-966b-401f0e0acb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.252784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccd472-5aa4-453e-966b-401f0e0acb72 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.263802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1269/2542] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:50.268682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.278963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d87e7-257a-41f1-a476-68f33f600333 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.283977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.284623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.284874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.285178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] PATH_INFO: `/v3/projects/f40252aa6a494b4daf10a6e3258c648f/users/52a0f3b3f7d54fdca6f5fe1cfb027c6c/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.303776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40252aa6a494b4daf10a6e3258c648f, user_id=52a0f3b3f7d54fdca6f5fe1cfb027c6c, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.306484 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.322682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1274/2543] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:50 2022] PUT /identity/v3/projects/f40252aa6a494b4daf10a6e3258c648f/users/52a0f3b3f7d54fdca6f5fe1cfb027c6c/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:50.334106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a497ef0-2859-4a78-8f91-1d5bcaf652d3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.334397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a497ef0-2859-4a78-8f91-1d5bcaf652d3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.334673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a497ef0-2859-4a78-8f91-1d5bcaf652d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.364228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5a497ef0-2859-4a78-8f91-1d5bcaf652d3 None None] MFA Rules not processed for user `52a0f3b3f7d54fdca6f5fe1cfb027c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:50.365477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5a497ef0-2859-4a78-8f91-1d5bcaf652d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:50.438565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1270/2544] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:50.443317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.521576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de987-b3f8-41df-bc53-5fc8e391432f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.529571 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.530339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.530689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.531058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.531769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.535133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.566991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1275/2545] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:50 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:00:50.570712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a497ef0-2859-4a78-8f91-1d5bcaf652d3 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.588085 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a497ef0-2859-4a78-8f91-1d5bcaf652d3 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.638737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.639376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.639650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.639937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.640573 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.642484 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.664647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1271/2546] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:50 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:50.668885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.679186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a266-2d45-41e1-b640-9b01cbe5b958 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.684061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.684660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.684939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.685205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.685879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.687466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.698070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1276/2547] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:00:50.702450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.713469 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aea0b2-f768-46fa-9788-d1398b8d08af admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.720990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.722128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.722542 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.722872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] PATH_INFO: `/v3/projects/5ef58ee321fe4d0d9692eea35ae782be/users/bcfa4a774d06419e8ae9ef82a460bcd5/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.741512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef58ee321fe4d0d9692eea35ae782be, user_id=bcfa4a774d06419e8ae9ef82a460bcd5, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.744527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.760315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1272/2548] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:50 2022] PUT /identity/v3/projects/5ef58ee321fe4d0d9692eea35ae782be/users/bcfa4a774d06419e8ae9ef82a460bcd5/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:50.764642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.774514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5bbc95-9d69-4986-8428-e3f1f7e0ebbf admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.823411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:50.831547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff136e-5095-4262-b8ad-c8d9b67ac578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.832190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff136e-5095-4262-b8ad-c8d9b67ac578 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.832468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff136e-5095-4262-b8ad-c8d9b67ac578 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.833213 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff136e-5095-4262-b8ad-c8d9b67ac578 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.834513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff136e-5095-4262-b8ad-c8d9b67ac578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.836384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff136e-5095-4262-b8ad-c8d9b67ac578 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.846667 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1277/2549] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:50.850927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:50.861366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18420d-b84b-45b0-b6ed-172488e4b764 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:50.866204 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:50.866842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.867115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.867365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] PATH_INFO: `/v3/projects/5ef58ee321fe4d0d9692eea35ae782be/users/bcfa4a774d06419e8ae9ef82a460bcd5/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.869718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef58ee321fe4d0d9692eea35ae782be, user_id=bcfa4a774d06419e8ae9ef82a460bcd5, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:50.872440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:50.883988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1273/2550] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:50 2022] PUT /identity/v3/projects/5ef58ee321fe4d0d9692eea35ae782be/users/bcfa4a774d06419e8ae9ef82a460bcd5/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:50.894205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:50.894492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:50.894768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:50.922784 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] MFA Rules not processed for user `bcfa4a774d06419e8ae9ef82a460bcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:50.924001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:50.993393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1278/2551] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:51.005872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:51.007639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:51.033109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:51.072508 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae0247-73ed-4dc8-b5eb-8579df483bc7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:51.076266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:51.076882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:51.077160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:51.077432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:51.086242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:51.087784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:51.096995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1274/2552] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:51.142823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:51.144479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:51.156731 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaafc5-8311-4205-a85d-433f38426433 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:51.160594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d47b5-9e1e-4197-ab69-4d23a0e7be0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:51.161212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d47b5-9e1e-4197-ab69-4d23a0e7be0b service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:51.161480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d47b5-9e1e-4197-ab69-4d23a0e7be0b service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:51.161743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d47b5-9e1e-4197-ab69-4d23a0e7be0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:51.220554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d47b5-9e1e-4197-ab69-4d23a0e7be0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:51.221984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d47b5-9e1e-4197-ab69-4d23a0e7be0b service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:51.231184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1279/2553] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:54.228992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:54.305477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd60252-a746-4d4d-a58e-56a6b98b29fd service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:54.310907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:54.311537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:54.311815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:54.312124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] PATH_INFO: `/v3/users/742eb4bbe39346b991cf316cbb329e23` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:54.313315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] RBAC: Authorizing `identity:delete_user(user_id=742eb4bbe39346b991cf316cbb329e23)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:54.315234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:54.366097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742eb4bbe39346b991cf316cbb329e23'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.366768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742eb4bbe39346b991cf316cbb329e23'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.374294 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742eb4bbe39346b991cf316cbb329e23'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.374944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742eb4bbe39346b991cf316cbb329e23'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.387458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742eb4bbe39346b991cf316cbb329e23'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.402526 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1275/2554] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:54 2022] DELETE /identity/v3/users/742eb4bbe39346b991cf316cbb329e23 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:54.407336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:54.407827 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:54.407988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:54.427394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:54.428836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:54.488913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1280/2555] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:54.499237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:54.500934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:54.548191 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:54.562291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af6361-2d4a-4c0a-b487-314bc33c865c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:54.566347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:54.566954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:54.567234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:54.567525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:54.576725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:54.578162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:54.587558 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1276/2556] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:54.861085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:54.862651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:54.875320 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7fde7-0213-4591-9e6f-d44b45ef6f4a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:54.883697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:54.884491 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:54.884776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:54.885054 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] PATH_INFO: `/v3/projects/7196c12c54b6476c816970eea2aa53d0` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:54.886245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7196c12c54b6476c816970eea2aa53d0)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:54.888494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:54.899877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:54.942560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7196c12c54b6476c816970eea2aa53d0', 'request_id': 'req-ff96cd8f-bf98-4037-b80d-d3998bb8a147'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.950146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7196c12c54b6476c816970eea2aa53d0', 'request_id': 'req-ff96cd8f-bf98-4037-b80d-d3998bb8a147'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.956875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7196c12c54b6476c816970eea2aa53d0', 'request_id': 'req-ff96cd8f-bf98-4037-b80d-d3998bb8a147'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.972323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] The token cache is being invalidate because project 7196c12c54b6476c816970eea2aa53d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:54.972653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.973258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:54.974959 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1281/2557] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:54 2022] DELETE /identity/v3/projects/7196c12c54b6476c816970eea2aa53d0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:54.979103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:54.989513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae95ca-cd2f-4dfa-a7c7-f0064dafbd31 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:55.044380 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:55.045063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:55.045343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:55.045615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] PATH_INFO: `/v3/users/52a0f3b3f7d54fdca6f5fe1cfb027c6c` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:55.046851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a0f3b3f7d54fdca6f5fe1cfb027c6c)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:55.048810 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:55.096431 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a0f3b3f7d54fdca6f5fe1cfb027c6c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.097074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a0f3b3f7d54fdca6f5fe1cfb027c6c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.104103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a0f3b3f7d54fdca6f5fe1cfb027c6c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.104718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a0f3b3f7d54fdca6f5fe1cfb027c6c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.117688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a0f3b3f7d54fdca6f5fe1cfb027c6c'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.134241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1277/2558] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:54 2022] DELETE /identity/v3/users/52a0f3b3f7d54fdca6f5fe1cfb027c6c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:55.427531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:55.429236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:55.449172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96cd8f-bf98-4037-b80d-d3998bb8a147 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:55.512450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:55.513114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:55.513387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:55.513807 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] PATH_INFO: `/v3/projects/f40252aa6a494b4daf10a6e3258c648f` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:55.515007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40252aa6a494b4daf10a6e3258c648f)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:55.516967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:55.573775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40252aa6a494b4daf10a6e3258c648f', 'request_id': 'req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.581389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40252aa6a494b4daf10a6e3258c648f', 'request_id': 'req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.588038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40252aa6a494b4daf10a6e3258c648f', 'request_id': 'req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.602832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] The token cache is being invalidate because project f40252aa6a494b4daf10a6e3258c648f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:55.603209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.603902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.605468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1282/2559] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:55 2022] DELETE /identity/v3/projects/f40252aa6a494b4daf10a6e3258c648f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:00:55.609745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:55.611167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:55.625009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf08d4-42be-4a11-9b69-4b9970bd3aed admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:55.680828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:55.681089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:55.681346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:55.681613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] PATH_INFO: `/v3/users/bcfa4a774d06419e8ae9ef82a460bcd5` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:55.682718 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcfa4a774d06419e8ae9ef82a460bcd5)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:55.684667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:55.689743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:55.743575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcfa4a774d06419e8ae9ef82a460bcd5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.743575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcfa4a774d06419e8ae9ef82a460bcd5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.747614 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcfa4a774d06419e8ae9ef82a460bcd5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.748507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcfa4a774d06419e8ae9ef82a460bcd5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.763313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-ccf28c26-3cf6-4c27-b982-6902ca802b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcfa4a774d06419e8ae9ef82a460bcd5'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:55.776424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1278/2560] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:00:55 2022] DELETE /identity/v3/users/bcfa4a774d06419e8ae9ef82a460bcd5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:56.054866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.056691 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:56.070209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb23c4e-bd56-4ae5-a65b-42071fbe34a7 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.099693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:56.135439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.136025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.136305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.136584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] PATH_INFO: `/v3/projects/5ef58ee321fe4d0d9692eea35ae782be` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.137714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ef58ee321fe4d0d9692eea35ae782be)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.139725 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.186693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ef58ee321fe4d0d9692eea35ae782be', 'request_id': 'req-0d7a2d79-d3c1-4d9c-8e01-214e34232677'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:56.193439 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ef58ee321fe4d0d9692eea35ae782be', 'request_id': 'req-0d7a2d79-d3c1-4d9c-8e01-214e34232677'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:56.199132 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ef58ee321fe4d0d9692eea35ae782be', 'request_id': 'req-0d7a2d79-d3c1-4d9c-8e01-214e34232677'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:56.213445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] The token cache is being invalidate because project 5ef58ee321fe4d0d9692eea35ae782be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:00:56.213734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:56.214343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0d7a2d79-d3c1-4d9c-8e01-214e34232677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:00:56.216069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1283/2561] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:00:56 2022] DELETE /identity/v3/projects/5ef58ee321fe4d0d9692eea35ae782be => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:00:56.222152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17a94-bc84-4bbf-91fb-a763ac908826 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.222444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17a94-bc84-4bbf-91fb-a763ac908826 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.222720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae17a94-bc84-4bbf-91fb-a763ac908826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.248765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2ae17a94-bc84-4bbf-91fb-a763ac908826 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:56.249984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2ae17a94-bc84-4bbf-91fb-a763ac908826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:56.312673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1279/2562] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:56.321514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49af4b-3498-4ad4-92e3-53c98e9aa6f3 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.321802 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49af4b-3498-4ad4-92e3-53c98e9aa6f3 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.322069 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49af4b-3498-4ad4-92e3-53c98e9aa6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.343367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5e49af4b-3498-4ad4-92e3-53c98e9aa6f3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:56.344605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5e49af4b-3498-4ad4-92e3-53c98e9aa6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:56.354874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1284/2563] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:56.360506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae17a94-bc84-4bbf-91fb-a763ac908826 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.375681 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae17a94-bc84-4bbf-91fb-a763ac908826 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.383962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.384836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.385158 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.385559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.386410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:00:56.386803 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.389071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.401388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1280/2564] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:00:56.405967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49af4b-3498-4ad4-92e3-53c98e9aa6f3 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.415493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49af4b-3498-4ad4-92e3-53c98e9aa6f3 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.420737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.421326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.421601 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.421872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.422499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.424414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.455075 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1285/2565] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:00:56 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:56.459601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.469666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed108a3b-ca58-4ba3-b23c-8f5cec495990 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.527738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.528391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.528673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.528945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.529566 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.531535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.547986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:56.553369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1281/2566] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:00:56 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:00:56.557684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.568117 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c7573-e19e-495a-ad98-3432cfd84b40 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.572918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.573506 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.573815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.574070 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.574709 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.576401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.588338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1286/2567] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:00:56.592523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.602954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68588ff-115c-49f4-8ec8-66362731fa3c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.608374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba61918-fe81-44f8-b8af-04b927c5e346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.608971 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba61918-fe81-44f8-b8af-04b927c5e346 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.609244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba61918-fe81-44f8-b8af-04b927c5e346 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.609510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba61918-fe81-44f8-b8af-04b927c5e346 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.610151 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba61918-fe81-44f8-b8af-04b927c5e346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.611749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba61918-fe81-44f8-b8af-04b927c5e346 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.623197 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1282/2568] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:00:56.627723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.642704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af03f9-0196-484a-98ae-07bb2693c87d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.651076 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.651991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.652362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.652690 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] PATH_INFO: `/v3/projects/30248b0e9f34461abd723ffbc1be4f40/users/2e21b7e6e61948c4b24ea671b1870f25/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.669025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30248b0e9f34461abd723ffbc1be4f40, user_id=2e21b7e6e61948c4b24ea671b1870f25, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.671716 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.689131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1287/2569] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:00:56 2022] PUT /identity/v3/projects/30248b0e9f34461abd723ffbc1be4f40/users/2e21b7e6e61948c4b24ea671b1870f25/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:00:56.701491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa524-bd1d-43ea-bf5e-de2b470d4f7e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.701775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa524-bd1d-43ea-bf5e-de2b470d4f7e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.702045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa524-bd1d-43ea-bf5e-de2b470d4f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.731517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0b4fa524-bd1d-43ea-bf5e-de2b470d4f7e None None] MFA Rules not processed for user `2e21b7e6e61948c4b24ea671b1870f25`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:00:56.732772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0b4fa524-bd1d-43ea-bf5e-de2b470d4f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:56.802787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1283/2570] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:00:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:00:56.815441 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:56.817543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:56.890436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4bc9e-2704-4bb2-9785-ab2307534f93 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:56.895325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f2313-08bf-4065-b97d-83a66f82b111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:56.895859 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2313-08bf-4065-b97d-83a66f82b111 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:56.896194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2313-08bf-4065-b97d-83a66f82b111 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:56.896471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2313-08bf-4065-b97d-83a66f82b111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:56.906196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f2313-08bf-4065-b97d-83a66f82b111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:56.907661 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f2313-08bf-4065-b97d-83a66f82b111 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:56.911207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:56.917702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1288/2571] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:05.466366 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fa524-bd1d-43ea-bf5e-de2b470d4f7e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:05.531021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fa524-bd1d-43ea-bf5e-de2b470d4f7e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:05.536053 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:05.536683 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:05.536961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:05.537249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] PATH_INFO: `/v3/users/2e21b7e6e61948c4b24ea671b1870f25` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:05.538394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e21b7e6e61948c4b24ea671b1870f25)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:05.540439 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:05.595688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e21b7e6e61948c4b24ea671b1870f25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:05.596377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e21b7e6e61948c4b24ea671b1870f25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:05.603467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e21b7e6e61948c4b24ea671b1870f25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:05.604157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e21b7e6e61948c4b24ea671b1870f25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:05.616596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e21b7e6e61948c4b24ea671b1870f25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:05.628553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:05.634216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1284/2572] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:01:05 2022] DELETE /identity/v3/users/2e21b7e6e61948c4b24ea671b1870f25 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:05.639084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:05.639670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:05.639771 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:05.659419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:05.660669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:05.718382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1289/2573] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:05.728621 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:05.730136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:05.798106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acc06c-2f93-4fc9-a8b1-040b9749892c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:05.802376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba3b5b-a730-4a10-a10e-f3e9d6270bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:05.802986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba3b5b-a730-4a10-a10e-f3e9d6270bcc service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:05.803265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba3b5b-a730-4a10-a10e-f3e9d6270bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:05.803534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba3b5b-a730-4a10-a10e-f3e9d6270bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:05.813370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba3b5b-a730-4a10-a10e-f3e9d6270bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:05.814802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba3b5b-a730-4a10-a10e-f3e9d6270bcc service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:05.824448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1285/2574] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:06.163012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.164510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:06.177083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d0623-f225-47bc-8025-1316567ce557 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.182747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.183395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.183672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.184101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] PATH_INFO: `/v3/projects/30248b0e9f34461abd723ffbc1be4f40` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.185394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=30248b0e9f34461abd723ffbc1be4f40)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.187381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:06.239333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30248b0e9f34461abd723ffbc1be4f40', 'request_id': 'req-1750dc9a-377c-49dd-9158-4515f245f3dd'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:06.245985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30248b0e9f34461abd723ffbc1be4f40', 'request_id': 'req-1750dc9a-377c-49dd-9158-4515f245f3dd'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:06.252048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30248b0e9f34461abd723ffbc1be4f40', 'request_id': 'req-1750dc9a-377c-49dd-9158-4515f245f3dd'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:06.266207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] The token cache is being invalidate because project 30248b0e9f34461abd723ffbc1be4f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:01:06.266552 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:06.267169 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1750dc9a-377c-49dd-9158-4515f245f3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:06.268659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1290/2575] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:01:06 2022] DELETE /identity/v3/projects/30248b0e9f34461abd723ffbc1be4f40 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:06.273695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e501-c81d-4cb5-bb05-41cc3b8ab6fe None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.273973 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e501-c81d-4cb5-bb05-41cc3b8ab6fe None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.274236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9e501-c81d-4cb5-bb05-41cc3b8ab6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.294717 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-27b9e501-c81d-4cb5-bb05-41cc3b8ab6fe None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:06.295918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-27b9e501-c81d-4cb5-bb05-41cc3b8ab6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:06.363064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1286/2576] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:06.371357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931cc96-ff7f-4da3-8445-71f918b12709 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.371673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931cc96-ff7f-4da3-8445-71f918b12709 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.371976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931cc96-ff7f-4da3-8445-71f918b12709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.392693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4931cc96-ff7f-4da3-8445-71f918b12709 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:06.393898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4931cc96-ff7f-4da3-8445-71f918b12709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:06.404835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1291/2577] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:06.409286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e501-c81d-4cb5-bb05-41cc3b8ab6fe None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.419696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9e501-c81d-4cb5-bb05-41cc3b8ab6fe None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.425746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.426349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.426620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.426885 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.427531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:01:06.427831 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.429458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:06.441776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1287/2578] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:01:06.446269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931cc96-ff7f-4da3-8445-71f918b12709 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.455918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931cc96-ff7f-4da3-8445-71f918b12709 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.461710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.462301 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.462622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.462868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.463509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.465422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:06.504426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1292/2579] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:01:06 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:01:06.508672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.519163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22081c0-0b17-4ce8-b235-4ffba9806e6a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.575374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.576014 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.576291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.576555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.577186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.579099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:06.602496 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1288/2580] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:01:06 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:01:06.606809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.617145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c143-2e0d-4ccf-b744-db78d135694b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.622679 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.623277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.623551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.623817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.624483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.626126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:06.635926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1293/2581] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:06.639805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.650554 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71718b41-82bc-47fb-a6df-8405f5c50749 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.656103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7d8c1-e19a-4341-b800-af09093b42a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.656772 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7d8c1-e19a-4341-b800-af09093b42a9 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.657048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7d8c1-e19a-4341-b800-af09093b42a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.657301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7d8c1-e19a-4341-b800-af09093b42a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.657939 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7d8c1-e19a-4341-b800-af09093b42a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.659539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7d8c1-e19a-4341-b800-af09093b42a9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:06.672589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1289/2582] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:06.678158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.689397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fc938-27e3-46a2-a909-50dfd182f04d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.695693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.696385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.696664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.696963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] PATH_INFO: `/v3/projects/2e6c21ca88dc48e2a134bbd5e84467de/users/76f82131c7a043df8e17cac16369b19c/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.714288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6c21ca88dc48e2a134bbd5e84467de, user_id=76f82131c7a043df8e17cac16369b19c, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.717414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:06.731368 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1294/2583] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:01:06 2022] PUT /identity/v3/projects/2e6c21ca88dc48e2a134bbd5e84467de/users/76f82131c7a043df8e17cac16369b19c/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:06.800461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7f5ff-a9df-4528-97a9-e0a26720f95b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.800728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7f5ff-a9df-4528-97a9-e0a26720f95b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.800974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7f5ff-a9df-4528-97a9-e0a26720f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.815159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:06.831290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8fd7f5ff-a9df-4528-97a9-e0a26720f95b None None] MFA Rules not processed for user `76f82131c7a043df8e17cac16369b19c`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:06.832524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8fd7f5ff-a9df-4528-97a9-e0a26720f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:06.898155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1290/2584] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:06.902896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:06.924254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:06.967778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d810b7-0dfa-462c-91cf-45e5db26cb90 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:06.972726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:06.973337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:06.973615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:06.973879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:06.974514 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:06.976592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.006923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1295/2585] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:01:06 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:01:07.011203 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7f5ff-a9df-4528-97a9-e0a26720f95b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.021938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7f5ff-a9df-4528-97a9-e0a26720f95b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.081809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.082340 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.082612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.082878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.083521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.085468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.109296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1291/2586] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:01:07 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:01:07.113501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.124685 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93010b2a-a47f-474e-9386-2713b377cd2b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.129779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.130415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.130695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.130967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.131616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.133239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.144380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1296/2587] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:07.148710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.160549 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d6462-83d4-4f33-8589-bd7ed2c320d1 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.165802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd88473-71c7-4fbf-a24a-c65fae06f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.166538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd88473-71c7-4fbf-a24a-c65fae06f41c admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.166815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd88473-71c7-4fbf-a24a-c65fae06f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.167074 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd88473-71c7-4fbf-a24a-c65fae06f41c admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.167715 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd88473-71c7-4fbf-a24a-c65fae06f41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.169346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd88473-71c7-4fbf-a24a-c65fae06f41c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.180243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1292/2588] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:07.184385 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.194543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50afff-44f4-4749-9818-a96c53080ac2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.199431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.200173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.200418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.200713 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] PATH_INFO: `/v3/projects/08cfbcccbe35487d98cca92bbd2f013a/users/2c6fded8e3d94677a29bef87c9e8c321/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.215686 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cfbcccbe35487d98cca92bbd2f013a, user_id=2c6fded8e3d94677a29bef87c9e8c321, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.218344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.234393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1297/2589] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:01:07 2022] PUT /identity/v3/projects/08cfbcccbe35487d98cca92bbd2f013a/users/2c6fded8e3d94677a29bef87c9e8c321/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:01:07.245902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b342-6dc9-40e0-a1b1-e6658c4077fc None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.246244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b342-6dc9-40e0-a1b1-e6658c4077fc None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.246638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6b342-6dc9-40e0-a1b1-e6658c4077fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.286612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-16c6b342-6dc9-40e0-a1b1-e6658c4077fc None None] MFA Rules not processed for user `2c6fded8e3d94677a29bef87c9e8c321`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:07.288097 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-16c6b342-6dc9-40e0-a1b1-e6658c4077fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:07.354415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1293/2590] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:07.359134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.424727 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb96ad5-7b72-4d56-933b-2a88170d4bf4 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.430736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.431358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.431660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.431920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.432598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.434493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.469501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1298/2591] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:01:07 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:01:07.475029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6b342-6dc9-40e0-a1b1-e6658c4077fc None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.482022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:07.487359 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6b342-6dc9-40e0-a1b1-e6658c4077fc None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.554312 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.554952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.555189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.555462 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.556110 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.558015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.580521 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1294/2592] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:01:07 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:01:07.584783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.595170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9250b1-158d-42c9-aeef-2aff124daa0f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.600866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.601466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.601742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.602014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.602680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.604321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.615634 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1299/2593] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:07.619959 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.630390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb880ee8-183b-49b8-823c-67809167bfe3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.636092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.636753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.637032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.637321 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] PATH_INFO: `/v3/projects/d5b27f735cfa456dbe0208284df29108/users/3d950b5fafac46fc950f283d3fe21c44/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.654228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b27f735cfa456dbe0208284df29108, user_id=3d950b5fafac46fc950f283d3fe21c44, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.656936 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.672175 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1295/2594] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:01:07 2022] PUT /identity/v3/projects/d5b27f735cfa456dbe0208284df29108/users/3d950b5fafac46fc950f283d3fe21c44/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:01:07.676131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.686023 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8624c-1987-431d-9f2b-2c53951eb53f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.730202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:07.750232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a337d3-d82a-455b-a2a1-6bb5634ee920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.750839 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a337d3-d82a-455b-a2a1-6bb5634ee920 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.751104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a337d3-d82a-455b-a2a1-6bb5634ee920 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.751391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a337d3-d82a-455b-a2a1-6bb5634ee920 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.752084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a337d3-d82a-455b-a2a1-6bb5634ee920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.753671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a337d3-d82a-455b-a2a1-6bb5634ee920 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.763442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1300/2595] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:07.767815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.778382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78790be-3322-44a0-b862-32a2c405c1d7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:07.783481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:07.784161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.784440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.784712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] PATH_INFO: `/v3/projects/d5b27f735cfa456dbe0208284df29108/users/3d950b5fafac46fc950f283d3fe21c44/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.786664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b27f735cfa456dbe0208284df29108, user_id=3d950b5fafac46fc950f283d3fe21c44, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:07.789370 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:07.799801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1296/2596] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:01:07 2022] PUT /identity/v3/projects/d5b27f735cfa456dbe0208284df29108/users/3d950b5fafac46fc950f283d3fe21c44/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:01:07.810317 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:07.810605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:07.810877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:07.840251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] MFA Rules not processed for user `3d950b5fafac46fc950f283d3fe21c44`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:07.841450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:07.917263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1301/2597] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:07.933680 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:07.936059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:08.000493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89525a-f077-450f-aa78-f0ec9bd90854 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:08.004823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:08.005435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:08.005710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:08.005976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:08.067276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:08.068759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:08.077816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1297/2598] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:09.631638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:09.633271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:09.645893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe1e99-481b-44d3-9a2e-0815c5e30e6d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:09.650342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:09.650909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:09.651191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:09.651448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:09.712278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:09.713476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:09.722783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1302/2599] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:10.275919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:10.286583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e14735-55f5-473d-a5a8-4daddcc55cb4 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:10.290547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf895-79ea-4e2c-a673-8abcf02423b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:10.291143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf895-79ea-4e2c-a673-8abcf02423b6 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:10.291414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf895-79ea-4e2c-a673-8abcf02423b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:10.291682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf895-79ea-4e2c-a673-8abcf02423b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:10.300982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf895-79ea-4e2c-a673-8abcf02423b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:10.302411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf895-79ea-4e2c-a673-8abcf02423b6 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:10.306231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:10.313404 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1298/2600] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:15.299336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:15.314534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:15.314975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:15.315086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:15.344616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:15.345905 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:15.365774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b69d4-47bb-490e-ab69-1ed93a906bd9 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:15.371097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:15.371822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:15.372161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:15.372426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] PATH_INFO: `/v3/users/76f82131c7a043df8e17cac16369b19c` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:15.373547 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f82131c7a043df8e17cac16369b19c)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:15.375498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:15.378080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:15.409460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1299/2601] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 09:01:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:15.431590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f82131c7a043df8e17cac16369b19c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:15.432323 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f82131c7a043df8e17cac16369b19c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:15.436473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f82131c7a043df8e17cac16369b19c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:15.445379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f82131c7a043df8e17cac16369b19c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:15.460742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:15.461307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-103dc06a-bdd7-487c-97a7-1024f30403b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f82131c7a043df8e17cac16369b19c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:15.462861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:15.477944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1303/2602] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:01:15 2022] DELETE /identity/v3/users/76f82131c7a043df8e17cac16369b19c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:15.482269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:15.482562 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:15.482838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:15.504533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:15.505664 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:15.541282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34d2a8-a87c-4637-89fa-437d39ffcc9e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:15.580126 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1304/2603] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:01:15.592626 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:15.594058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:15.622625 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:15.623257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:15.623535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:15.623806 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:15.625656 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:15.651253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39368a-b37d-47f6-9fc3-27360d830fde None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:15.655253 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:15.655858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:15.656168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:15.656435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:15.665419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:15.665646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:15.667067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:15.676834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1305/2604] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 09:01:15.686449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:15.688073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:15.696468 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1300/2605] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:01:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 09:01:15.966095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:15.967763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:15.992738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf614b8-ca23-4195-8696-c3f8e2db5571 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:15.999985 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:16.000712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:16.001058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:16.001472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] PATH_INFO: `/v3/projects/2e6c21ca88dc48e2a134bbd5e84467de` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:16.002873 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6c21ca88dc48e2a134bbd5e84467de)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:16.005280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:16.027907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:16.057338 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6c21ca88dc48e2a134bbd5e84467de', 'request_id': 'req-eefc07a9-8d2c-4d46-9330-ebfd326e326b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.063676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6c21ca88dc48e2a134bbd5e84467de', 'request_id': 'req-eefc07a9-8d2c-4d46-9330-ebfd326e326b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.069415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6c21ca88dc48e2a134bbd5e84467de', 'request_id': 'req-eefc07a9-8d2c-4d46-9330-ebfd326e326b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.084707 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] The token cache is being invalidate because project 2e6c21ca88dc48e2a134bbd5e84467de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:01:16.085082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.085822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.087336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1306/2606] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:01:15 2022] DELETE /identity/v3/projects/2e6c21ca88dc48e2a134bbd5e84467de => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:16.091500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:16.102750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e15ceef-521f-4e69-8c25-036a140034fa service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:16.164534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:16.165267 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:16.165626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:16.165960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] PATH_INFO: `/v3/users/2c6fded8e3d94677a29bef87c9e8c321` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:16.167292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6fded8e3d94677a29bef87c9e8c321)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:16.169589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:16.221745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6fded8e3d94677a29bef87c9e8c321'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.222528 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6fded8e3d94677a29bef87c9e8c321'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.229745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6fded8e3d94677a29bef87c9e8c321'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.230410 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6fded8e3d94677a29bef87c9e8c321'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.246591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6fded8e3d94677a29bef87c9e8c321'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.262602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1301/2607] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:01:16 2022] DELETE /identity/v3/users/2c6fded8e3d94677a29bef87c9e8c321 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:16.557594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:16.559739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:16.602599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc07a9-8d2c-4d46-9330-ebfd326e326b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:16.724467 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:16.725487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:16.725651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:16.726150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] PATH_INFO: `/v3/projects/08cfbcccbe35487d98cca92bbd2f013a` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:16.727397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cfbcccbe35487d98cca92bbd2f013a)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:16.729519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:16.790299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cfbcccbe35487d98cca92bbd2f013a', 'request_id': 'req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.799087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cfbcccbe35487d98cca92bbd2f013a', 'request_id': 'req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.806379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cfbcccbe35487d98cca92bbd2f013a', 'request_id': 'req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.828180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] The token cache is being invalidate because project 08cfbcccbe35487d98cca92bbd2f013a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:01:16.828734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.829498 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:16.831592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1307/2608] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:01:16 2022] DELETE /identity/v3/projects/08cfbcccbe35487d98cca92bbd2f013a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:16.835596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:16.851687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:16.865941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10306b-0434-41b1-a565-d4948a5d01d9 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:16.928242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:16.937275 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:16.937927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:16.938215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:16.938559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] PATH_INFO: `/v3/users/3d950b5fafac46fc950f283d3fe21c44` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:16.940163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d950b5fafac46fc950f283d3fe21c44)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:16.942587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:17.009414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d950b5fafac46fc950f283d3fe21c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.010342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d950b5fafac46fc950f283d3fe21c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.018759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d950b5fafac46fc950f283d3fe21c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.019405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d950b5fafac46fc950f283d3fe21c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.032172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1dc41b5-fde4-4987-a2c7-b321233d5896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d950b5fafac46fc950f283d3fe21c44'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.048501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1302/2609] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:01:16 2022] DELETE /identity/v3/users/3d950b5fafac46fc950f283d3fe21c44 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:17.344627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:17.346189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:17.368112 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78955fc8-d78c-4804-b4c0-ac4f8428e9ff admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:17.407651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:17.434424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:17.435078 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:17.435356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:17.435635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] PATH_INFO: `/v3/projects/d5b27f735cfa456dbe0208284df29108` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:17.436850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b27f735cfa456dbe0208284df29108)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:17.438834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:17.494294 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b27f735cfa456dbe0208284df29108', 'request_id': 'req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.503543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b27f735cfa456dbe0208284df29108', 'request_id': 'req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.509636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b27f735cfa456dbe0208284df29108', 'request_id': 'req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.534024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] The token cache is being invalidate because project d5b27f735cfa456dbe0208284df29108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:01:17.534372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.534968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-42c1a3b0-f648-43ff-8f05-4de76ec1d16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:17.536742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1308/2610] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:01:17 2022] DELETE /identity/v3/projects/d5b27f735cfa456dbe0208284df29108 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:17.541553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e775a5-6540-4e6e-adb6-9f14c4218902 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:17.541887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e775a5-6540-4e6e-adb6-9f14c4218902 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:17.542201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e775a5-6540-4e6e-adb6-9f14c4218902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:17.565476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-f7e775a5-6540-4e6e-adb6-9f14c4218902 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:17.566701 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f7e775a5-6540-4e6e-adb6-9f14c4218902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:17.635098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1303/2611] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:17.644118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da01-875e-4879-a150-f968c0946dd2 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:17.644684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da01-875e-4879-a150-f968c0946dd2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:17.644684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da01-875e-4879-a150-f968c0946dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:17.664763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4bf7da01-875e-4879-a150-f968c0946dd2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:17.665991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4bf7da01-875e-4879-a150-f968c0946dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:17.675993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1309/2612] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:17.680363 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e775a5-6540-4e6e-adb6-9f14c4218902 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:17.692168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e775a5-6540-4e6e-adb6-9f14c4218902 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:17.697618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:17.698235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:17.698511 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:17.698778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:17.699429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:01:17.699727 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:17.701698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:17.717052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1304/2613] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:01:17.722133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7da01-875e-4879-a150-f968c0946dd2 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:17.733930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7da01-875e-4879-a150-f968c0946dd2 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:17.738986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:17.739614 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:17.739920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:17.740202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:17.740845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:17.742905 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:17.777354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1310/2614] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:01:17 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:01:17.778525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:17.788979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe649af-43d5-41bd-9cd3-0393054f6d8f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:17.869795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:17.870463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:17.870708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:17.870979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:17.871636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:17.873622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:17.900837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:17.923824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1305/2615] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:01:17 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:01:17.924776 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:17.940789 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd0aae-2ae8-4a88-bea7-5f0846773b37 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:17.961583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:17.962529 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:17.963008 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:17.963391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:17.964542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:17.978650 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:18.001874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1311/2616] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:18.005871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:18.033911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e91e37-8606-43ab-851d-a372ba0fb317 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:18.040099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007be714-aa9b-4bb3-b947-f4273075e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:18.040759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007be714-aa9b-4bb3-b947-f4273075e9bf admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:18.041221 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007be714-aa9b-4bb3-b947-f4273075e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:18.041495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007be714-aa9b-4bb3-b947-f4273075e9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:18.042327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007be714-aa9b-4bb3-b947-f4273075e9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:18.044162 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007be714-aa9b-4bb3-b947-f4273075e9bf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:18.060624 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1306/2617] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:01:18.061896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:18.075438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bb22b-138d-4820-9b42-f921bda06615 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:18.083773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:18.084877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:18.085326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:18.085765 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] PATH_INFO: `/v3/projects/4cefdaaad09e451dbb9e5e735845a9cc/users/9d49791d656a4cbaaf30609253a0e565/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:18.106665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cefdaaad09e451dbb9e5e735845a9cc, user_id=9d49791d656a4cbaaf30609253a0e565, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:18.109779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:18.127073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1312/2618] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:01:18 2022] PUT /identity/v3/projects/4cefdaaad09e451dbb9e5e735845a9cc/users/9d49791d656a4cbaaf30609253a0e565/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:18.137781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062eff-6ff6-4e22-b1d5-4478de0a752e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:18.138070 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062eff-6ff6-4e22-b1d5-4478de0a752e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:18.138339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062eff-6ff6-4e22-b1d5-4478de0a752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:18.179441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9f062eff-6ff6-4e22-b1d5-4478de0a752e None None] MFA Rules not processed for user `9d49791d656a4cbaaf30609253a0e565`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:18.181002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9f062eff-6ff6-4e22-b1d5-4478de0a752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:18.261963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1307/2619] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:18.273831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:18.276001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:18.346151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f30d9-da89-403a-8fea-2fd75087ac5b admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:18.350139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:18.350845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:18.351218 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:18.351555 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:18.361095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:18.362628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:18.372838 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1313/2620] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:19.822937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:19.823517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:19.823517 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:19.843305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] MFA Rules not processed for user `9d49791d656a4cbaaf30609253a0e565`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:19.844744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:19.852886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1308/2621] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:28.368095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:29.838581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:INSERT=1,keystone:SELECT=42 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:31.634122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:31.643825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8984453-d0ae-4f7d-8b4f-ac3d0faca33a service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:31.648060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:31.648699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:31.648981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:31.649277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:31.651413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:31.744483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:31.745916 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:31.757794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1314/2622] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 09:01:32.452475 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:32.454038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:32.511224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faf41b-0b38-40b6-809c-2d6ab9ad7a8e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:32.514906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557974bcc89c412f8d5c2e2039843f65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:32.516515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:32.516796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:32.517066 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:32.518951 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:32.533077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:32.534514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:32.545550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1309/2623] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:32.612517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:32.613948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:32.670711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f270c8-bd60-41ac-8182-3841f173071b service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:32.674781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:32.675346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:32.675683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:32.675972 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:32.677871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:32.688201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:32.689628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:32.697567 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1315/2624] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:32.779384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:32.781115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:32.793743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de5b80-c040-4833-a523-f21bcffacf14 service glance] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:32.798207 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3902b9-d967-4b8b-83d4-1ea34d4f342e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:32.798802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3902b9-d967-4b8b-83d4-1ea34d4f342e service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:32.799086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3902b9-d967-4b8b-83d4-1ea34d4f342e service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:32.799347 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3902b9-d967-4b8b-83d4-1ea34d4f342e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:32.801210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9e3902b9-d967-4b8b-83d4-1ea34d4f342e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:32.814918 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3902b9-d967-4b8b-83d4-1ea34d4f342e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:32.816516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3902b9-d967-4b8b-83d4-1ea34d4f342e service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:32.828003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1310/2625] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:33.149549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:33.158454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d8310-203c-48e8-b509-5f074fc38426 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:33.162428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:33.163029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:33.163305 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:33.163573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:33.165452 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:33.178680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:33.180168 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:33.191239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1316/2626] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:36.592872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:36.593375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:36.593481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:36.613813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:36.615159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:36.670051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1311/2627] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 09:01:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:36.682678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:36.684155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:36.758746 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625ceba-cdc1-4c21-9395-3474924a444e service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:36.764036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9d9f1-7051-4b2c-9eb6-d60514c91718 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:36.764779 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9d9f1-7051-4b2c-9eb6-d60514c91718 service swift] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:36.765155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9d9f1-7051-4b2c-9eb6-d60514c91718 service swift] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:36.765521 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9d9f1-7051-4b2c-9eb6-d60514c91718 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:36.770849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:36.775448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9d9f1-7051-4b2c-9eb6-d60514c91718 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:36.777096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9d9f1-7051-4b2c-9eb6-d60514c91718 service swift] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:36.785996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1317/2628] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:01:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:01:45.593327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:45.594962 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:45.612224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9314-c5d5-4897-833f-b0aeca2fe1a8 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:45.619862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:45.620498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:45.620775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:45.621043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:45.622881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:45.706900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:45.708533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:45.717523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1312/2629] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:46.782248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:55.498855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b635327-76f8-41a6-99f6-e26578fe7501 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:55.499479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b635327-76f8-41a6-99f6-e26578fe7501 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:55.499479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b635327-76f8-41a6-99f6-e26578fe7501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:55.591619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4b635327-76f8-41a6-99f6-e26578fe7501 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:55.593158 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4b635327-76f8-41a6-99f6-e26578fe7501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:55.629405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1318/2630] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 09:01:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:55.637965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:55.679661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:55.682778 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb8d38-7342-4d69-99e4-bc19aa323cbb service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:55.689694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:55.690784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:55.691135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:55.691492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:55.701355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:55.703226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:55.713249 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1313/2631] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:01:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 09:01:56.346109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd40a8-c29e-4a57-b0c0-ce00c2e91dcf None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:56.346401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd40a8-c29e-4a57-b0c0-ce00c2e91dcf None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:56.346745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd40a8-c29e-4a57-b0c0-ce00c2e91dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:56.367165 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-54fd40a8-c29e-4a57-b0c0-ce00c2e91dcf None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:56.368454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-54fd40a8-c29e-4a57-b0c0-ce00c2e91dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:56.376737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1319/2632] 10.222.0.44 () {58 vars in 967 bytes} [Tue Oct 4 09:01:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:56.392956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:56.405082 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae287966-1604-4660-98a5-231f6e65590b service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:56.425408 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:56.426406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:56.427018 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:56.427470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:56.439398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:56.441530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:56.452640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1314/2633] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:01:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:01:56.699428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bce3f-eb4a-4a62-a249-d520314fda80 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:56.699944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bce3f-eb4a-4a62-a249-d520314fda80 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:56.700167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bce3f-eb4a-4a62-a249-d520314fda80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:56.719774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d61bce3f-eb4a-4a62-a249-d520314fda80 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:56.721045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d61bce3f-eb4a-4a62-a249-d520314fda80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:56.729747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1320/2634] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Oct 4 09:01:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:56.742906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:56.754098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2add-bb68-46d9-a498-c9316bb42b45 service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:56.759550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:56.760282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:56.760616 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:56.761079 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:56.769921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:56.771547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:56.780281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1315/2635] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:01:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:01:57.121976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147bbb7-951e-4647-b9e5-1e31060dace6 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:57.122262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147bbb7-951e-4647-b9e5-1e31060dace6 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:57.122537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147bbb7-951e-4647-b9e5-1e31060dace6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:57.143744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f147bbb7-951e-4647-b9e5-1e31060dace6 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:57.145038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f147bbb7-951e-4647-b9e5-1e31060dace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:57.153696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1321/2636] 10.222.0.44 () {58 vars in 967 bytes} [Tue Oct 4 09:01:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:57.167531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:57.178146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf0dab-02eb-4573-9aa5-8bb3a985ea37 service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:57.183286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c96397-70ca-4100-80af-4afeebfeeb27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:57.183907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c96397-70ca-4100-80af-4afeebfeeb27 service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:57.184310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c96397-70ca-4100-80af-4afeebfeeb27 service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:57.184580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c96397-70ca-4100-80af-4afeebfeeb27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:57.193069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c96397-70ca-4100-80af-4afeebfeeb27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:57.194799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c96397-70ca-4100-80af-4afeebfeeb27 service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:57.203285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1316/2637] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:01:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:01:59.338976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147bbb7-951e-4647-b9e5-1e31060dace6 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:59.348306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423ae23-975a-4b76-b75b-74f9c39e8f33 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:59.348788 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423ae23-975a-4b76-b75b-74f9c39e8f33 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:59.348901 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423ae23-975a-4b76-b75b-74f9c39e8f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:59.368516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-8423ae23-975a-4b76-b75b-74f9c39e8f33 None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:59.370057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8423ae23-975a-4b76-b75b-74f9c39e8f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:59.377444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1317/2638] 10.222.0.44 () {58 vars in 967 bytes} [Tue Oct 4 09:01:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:59.389710 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423ae23-975a-4b76-b75b-74f9c39e8f33 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:59.401563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423ae23-975a-4b76-b75b-74f9c39e8f33 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:59.402925 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147bbb7-951e-4647-b9e5-1e31060dace6 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:59.406875 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd35e-2600-410a-a85c-8921a8270998 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:59.407665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:59.407798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd35e-2600-410a-a85c-8921a8270998 service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:59.408130 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd35e-2600-410a-a85c-8921a8270998 service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:59.408322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:59.408499 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd35e-2600-410a-a85c-8921a8270998 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:59.408600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:59.408961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] PATH_INFO: `/v3/users/9d49791d656a4cbaaf30609253a0e565` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:59.410616 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d49791d656a4cbaaf30609253a0e565)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:59.413957 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:59.416922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd35e-2600-410a-a85c-8921a8270998 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:59.418491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd35e-2600-410a-a85c-8921a8270998 service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:59.426978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1318/2639] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:01:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 09:01:59.463247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d49791d656a4cbaaf30609253a0e565'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.463979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d49791d656a4cbaaf30609253a0e565'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.464576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d49791d656a4cbaaf30609253a0e565'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.472230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d49791d656a4cbaaf30609253a0e565'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.484947 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d49791d656a4cbaaf30609253a0e565'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.502862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1322/2640] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:01:59 2022] DELETE /identity/v3/users/9d49791d656a4cbaaf30609253a0e565 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:01:59.507699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:59.507932 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:59.508248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:59.528696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:01:59.529325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:59.594355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1319/2641] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:01:59.604391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:59.605908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:59.630730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:01:59.673233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5d0e1-86c7-4ec1-b831-3e41e242d7dc admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:59.677388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f56329-d5d7-45b2-9cc6-572df16b69a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:59.678001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56329-d5d7-45b2-9cc6-572df16b69a4 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:59.678289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56329-d5d7-45b2-9cc6-572df16b69a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:59.678733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56329-d5d7-45b2-9cc6-572df16b69a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:59.688912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f56329-d5d7-45b2-9cc6-572df16b69a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:59.690440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f56329-d5d7-45b2-9cc6-572df16b69a4 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:59.700509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1323/2642] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:01:59.874588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:01:59.876136 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:01:59.888577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb48244-ac80-4e7c-87dd-b51cc27dc694 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:01:59.893884 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:01:59.894531 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:01:59.894811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:01:59.895077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] PATH_INFO: `/v3/projects/4cefdaaad09e451dbb9e5e735845a9cc` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:01:59.896341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cefdaaad09e451dbb9e5e735845a9cc)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:01:59.898269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:01:59.947008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cefdaaad09e451dbb9e5e735845a9cc', 'request_id': 'req-83943004-902c-4225-8785-a2206793a0f4'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.953398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cefdaaad09e451dbb9e5e735845a9cc', 'request_id': 'req-83943004-902c-4225-8785-a2206793a0f4'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.959172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cefdaaad09e451dbb9e5e735845a9cc', 'request_id': 'req-83943004-902c-4225-8785-a2206793a0f4'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.975297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] The token cache is being invalidate because project 4cefdaaad09e451dbb9e5e735845a9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:01:59.975600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.976233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:01:59.977768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1320/2643] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:01:59 2022] DELETE /identity/v3/projects/4cefdaaad09e451dbb9e5e735845a9cc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:02:02.445059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:02.445591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:02.445591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:02.465649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] MFA Rules not processed for user `3e23eac1e2b04f59bbdac1646da01cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:02:02.466841 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:02.524488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1324/2644] 10.222.0.44 () {58 vars in 967 bytes} [Tue Oct 4 09:02:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:02:02.537093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:02.539271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:02.567762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:02.634671 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943004-902c-4225-8785-a2206793a0f4 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:02.643787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c76ff74c7e67426bb2b94a4bb1a2b9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:02.644464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:02.644773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:02.645080 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:02.658008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:02.659589 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:02.667935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1321/2645] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Oct 4 09:02:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:02:06.123148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:06.124735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:06.211193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81613854-8983-46dd-a4cb-b295ff17b66c None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:06.216212 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:06.216984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:06.217370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:06.217646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:06.219463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:06.231470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:06.233247 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:06.245122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1325/2646] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:09.018233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:09.029636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a9b9-0567-4e48-b7a7-957830d4b49c service swift] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:09.035560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:09.036226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:09.036490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:09.036756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:09.038460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:09.050828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:09.052639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:09.063150 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1322/2647] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:09.098792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:09.100367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:09.167665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:09.172812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099c7da-7e11-4d08-ad14-81e27970890e service placement] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:09.177684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:09.178321 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:09.178600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:09.178874 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] PATH_INFO: `/v3/users/3fb4f49a5be0475889369e13c3bc1481` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:09.180049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fb4f49a5be0475889369e13c3bc1481)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:09.181986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:09.236738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fb4f49a5be0475889369e13c3bc1481'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.237423 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fb4f49a5be0475889369e13c3bc1481'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.238060 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fb4f49a5be0475889369e13c3bc1481'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.245350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fb4f49a5be0475889369e13c3bc1481'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.257560 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fb4f49a5be0475889369e13c3bc1481'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.273906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1326/2648] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:02:09 2022] DELETE /identity/v3/users/3fb4f49a5be0475889369e13c3bc1481 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:02:09.450024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:09.462550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a25e71-31cc-449a-91a5-7542d973d4ff service placement] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:09.520503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:09.521149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:09.521452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:09.521719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] PATH_INFO: `/v3/projects/62a1577879c64dc6aac3d5fd4d7ad637` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:09.522857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a1577879c64dc6aac3d5fd4d7ad637)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:09.524840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:09.571490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a1577879c64dc6aac3d5fd4d7ad637', 'request_id': 'req-bd9231bf-30a7-4a10-bae6-b1d291b717bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.577796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a1577879c64dc6aac3d5fd4d7ad637', 'request_id': 'req-bd9231bf-30a7-4a10-bae6-b1d291b717bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.584173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a1577879c64dc6aac3d5fd4d7ad637', 'request_id': 'req-bd9231bf-30a7-4a10-bae6-b1d291b717bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.601254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] The token cache is being invalidate because project 62a1577879c64dc6aac3d5fd4d7ad637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:02:09.601538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.602432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.604785 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1323/2649] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:02:09 2022] DELETE /identity/v3/projects/62a1577879c64dc6aac3d5fd4d7ad637 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:02:09.610170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:09.611653 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:09.625977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7316-ed61-4fe2-81d5-a57b7b2a5861 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:09.683757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:09.684335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:09.684619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:09.684887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] PATH_INFO: `/v3/users/6fed61b47090479ba4b158a5033645d6` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:09.686037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fed61b47090479ba4b158a5033645d6)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:09.688019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:09.740379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fed61b47090479ba4b158a5033645d6'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.741025 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fed61b47090479ba4b158a5033645d6'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.741629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fed61b47090479ba4b158a5033645d6'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.750503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fed61b47090479ba4b158a5033645d6'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.766976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d50ab8c2-e6f9-40ee-bb50-71c6254ff1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fed61b47090479ba4b158a5033645d6'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:09.770684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:09.783454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1327/2650] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:02:09 2022] DELETE /identity/v3/users/6fed61b47090479ba4b158a5033645d6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:02:09.960677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:09.962361 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:09.980878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9231bf-30a7-4a10-bae6-b1d291b717bd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:09.989271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:10.042466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:10.043196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:10.043362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:10.043631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] PATH_INFO: `/v3/projects/0f35bffe6f7e4d899340dc4f9781b8c3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:10.044838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f35bffe6f7e4d899340dc4f9781b8c3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:10.046743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:10.101223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f35bffe6f7e4d899340dc4f9781b8c3', 'request_id': 'req-80e35b5d-082d-439a-8743-eb9b004d66c8'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:10.107916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f35bffe6f7e4d899340dc4f9781b8c3', 'request_id': 'req-80e35b5d-082d-439a-8743-eb9b004d66c8'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:10.115262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f35bffe6f7e4d899340dc4f9781b8c3', 'request_id': 'req-80e35b5d-082d-439a-8743-eb9b004d66c8'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:10.132041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] The token cache is being invalidate because project 0f35bffe6f7e4d899340dc4f9781b8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:02:10.132355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:10.133048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:02:10.134595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1324/2651] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:02:09 2022] DELETE /identity/v3/projects/0f35bffe6f7e4d899340dc4f9781b8c3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:02:19.784931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=2 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:20.128838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:41.030088 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493aed1-abed-4034-a6e4-69a8deb71947 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:41.030647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493aed1-abed-4034-a6e4-69a8deb71947 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:41.030647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493aed1-abed-4034-a6e4-69a8deb71947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:41.050754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d493aed1-abed-4034-a6e4-69a8deb71947 None None] MFA Rules not processed for user `d8d329978baf425e954b9965ae0cb652`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:02:41.052016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d493aed1-abed-4034-a6e4-69a8deb71947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:41.125239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1328/2652] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:02:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:02:44.201771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:44.203357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:44.269974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e35b5d-082d-439a-8743-eb9b004d66c8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:44.273602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:44.274258 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:44.274520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:44.274786 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:44.276828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:44.287816 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:44.289402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:44.298041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1325/2653] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:51.116130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=25 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:54.294544 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=26 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:58.743919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493aed1-abed-4034-a6e4-69a8deb71947 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:58.758012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493aed1-abed-4034-a6e4-69a8deb71947 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:58.761695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:58.762440 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:58.762721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:58.762993 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:58.765190 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:58.839609 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:58.839934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:58.849945 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1329/2654] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 09:02:59.008603 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:59.010093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:59.020704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e90d7-cfd1-44b8-a78b-3da81d2d8fae service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:59.024422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:59.025017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:59.025293 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:59.025565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:59.027275 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:59.090179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:59.091654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:59.101619 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1326/2655] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:03.008782 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.010570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:03.024643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f2e8-a1c9-42d5-bf71-bc73b9c5ed5a service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.029712 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.030397 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.030739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.031009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] PATH_INFO: `/v3/users/e71984f02fcd4deba5e1430ec85b8705` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.032863 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71984f02fcd4deba5e1430ec85b8705)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.034861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.088666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71984f02fcd4deba5e1430ec85b8705'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.089339 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71984f02fcd4deba5e1430ec85b8705'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.090267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71984f02fcd4deba5e1430ec85b8705'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.099009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71984f02fcd4deba5e1430ec85b8705'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.113669 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-10d5ff85-9484-4943-bb29-bbeafd78ff37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71984f02fcd4deba5e1430ec85b8705'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.129865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1330/2656] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:03 2022] DELETE /identity/v3/users/e71984f02fcd4deba5e1430ec85b8705 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:03.323968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.335598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d963-8983-4300-9bc2-d548a4dbee54 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.394732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.395279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.395552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.395811 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] PATH_INFO: `/v3/projects/b7ad37ea522247c6b28ea40e1bc43669` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.397093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ad37ea522247c6b28ea40e1bc43669)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.399002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.446709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ad37ea522247c6b28ea40e1bc43669', 'request_id': 'req-2cd46ea9-57bc-4681-9264-f254fe82e68b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.452822 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ad37ea522247c6b28ea40e1bc43669', 'request_id': 'req-2cd46ea9-57bc-4681-9264-f254fe82e68b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.459476 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ad37ea522247c6b28ea40e1bc43669', 'request_id': 'req-2cd46ea9-57bc-4681-9264-f254fe82e68b'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.474028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] The token cache is being invalidate because project b7ad37ea522247c6b28ea40e1bc43669 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:03.474394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.475038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-2cd46ea9-57bc-4681-9264-f254fe82e68b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:03.476577 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1327/2657] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:03 2022] DELETE /identity/v3/projects/b7ad37ea522247c6b28ea40e1bc43669 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:03.481794 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c3e54-a270-4c90-ae99-87812768573f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.482079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c3e54-a270-4c90-ae99-87812768573f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.482353 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c3e54-a270-4c90-ae99-87812768573f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.502413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4a0c3e54-a270-4c90-ae99-87812768573f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:03.503618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4a0c3e54-a270-4c90-ae99-87812768573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:03.580740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1331/2658] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:03.594994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637d17e-2158-4c9b-8a82-06ae571fc80e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.595413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637d17e-2158-4c9b-8a82-06ae571fc80e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.595817 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637d17e-2158-4c9b-8a82-06ae571fc80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.622244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-f637d17e-2158-4c9b-8a82-06ae571fc80e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:03.623451 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f637d17e-2158-4c9b-8a82-06ae571fc80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:03.633011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1328/2659] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:03.637531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c3e54-a270-4c90-ae99-87812768573f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.647923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c3e54-a270-4c90-ae99-87812768573f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.653252 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.653855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.654116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.654382 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.655027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:03.655336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.656965 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.658436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:03.671876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1332/2660] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:03.676069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637d17e-2158-4c9b-8a82-06ae571fc80e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.685160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637d17e-2158-4c9b-8a82-06ae571fc80e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.690103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.690690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.690965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.691234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.691857 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.693787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.709565 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:03.733244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1329/2661] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:03 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:03.737476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.748346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36ae47-3d0a-47c7-8ee1-833cffbdfad2 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.807696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.808191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.808450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.808728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.809375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.811302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.842031 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1333/2662] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:03 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:03.843471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.854690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaac7d-ebbf-408e-8b84-0931f1447614 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.859968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.860591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.860862 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.861148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.861763 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.863492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.879530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1330/2663] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:03.884766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.895638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88411077-8d6c-41f9-960f-2b05aaa1e1d8 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.901282 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a070a3-a93d-40eb-82fc-da111f2b12bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.901830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a070a3-a93d-40eb-82fc-da111f2b12bc admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.902129 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a070a3-a93d-40eb-82fc-da111f2b12bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.902335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a070a3-a93d-40eb-82fc-da111f2b12bc admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.902982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a070a3-a93d-40eb-82fc-da111f2b12bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.904628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a070a3-a93d-40eb-82fc-da111f2b12bc admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.916432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1334/2664] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:03.920682 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:03.931031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3ecbd-ab1c-43d0-ad7a-327ce137af39 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:03.935670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:03.936367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.936649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.936947 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] PATH_INFO: `/v3/projects/89b56ee603d34a5f85f7e007b2fe668f/users/3ec88da1704d44fb924fa3cc04e5fe25/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:03.952357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b56ee603d34a5f85f7e007b2fe668f, user_id=3ec88da1704d44fb924fa3cc04e5fe25, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:03.955008 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:03.974767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1331/2665] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:03 2022] PUT /identity/v3/projects/89b56ee603d34a5f85f7e007b2fe668f/users/3ec88da1704d44fb924fa3cc04e5fe25/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:03.980186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31a7f9-669f-4863-8c7f-494ec4d0af86 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:03.980473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31a7f9-669f-4863-8c7f-494ec4d0af86 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:03.980743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31a7f9-669f-4863-8c7f-494ec4d0af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:04.000383 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ac31a7f9-669f-4863-8c7f-494ec4d0af86 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:04.001704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ac31a7f9-669f-4863-8c7f-494ec4d0af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:04.060674 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1335/2666] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:04.070522 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:04.072374 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:04.131838 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4a14a-ba3f-4fc9-b34f-e97a536dbbaf admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:04.136667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:04.137411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:04.139434 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:04.139720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:04.149986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:04.151474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:04.161065 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1332/2667] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:08.984850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97f87c-b11a-49cf-b375-72dfff9fa6b0 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:08.985337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97f87c-b11a-49cf-b375-72dfff9fa6b0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:08.985426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97f87c-b11a-49cf-b375-72dfff9fa6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:09.025499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9b97f87c-b11a-49cf-b375-72dfff9fa6b0 None None] MFA Rules not processed for user `3ec88da1704d44fb924fa3cc04e5fe25`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:09.026982 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9b97f87c-b11a-49cf-b375-72dfff9fa6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:09.115488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1336/2668] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:09.136325 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:09.137960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:09.227791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12e5e-a66a-425d-b55c-48a4332e9e87 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:09.234356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:09.234987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:09.235264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:09.235538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:09.245238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:09.246915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:09.258290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1333/2669] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:11.613630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:11.614176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:11.614176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:11.622062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:11.636696 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] MFA Rules not processed for user `3ec88da1704d44fb924fa3cc04e5fe25`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:11.637910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:11.645205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1337/2670] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:12.038210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:12.041111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:12.063911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8d725-b257-4087-8ebc-e50f74ee6f98 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:12.067730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:12.068397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:12.068755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:12.069027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:12.070855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:12.088310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:12.090189 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:12.102593 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1334/2671] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:16.055977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.057551 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:16.073943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f262eec-5be9-4daf-8cc9-f49f6717a968 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.080033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.080684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.080960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.081254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] PATH_INFO: `/v3/users/68288f4e2f57454fbb06eff0dd72cca3` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.082613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68288f4e2f57454fbb06eff0dd72cca3)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.084984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:16.145769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68288f4e2f57454fbb06eff0dd72cca3'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.146589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68288f4e2f57454fbb06eff0dd72cca3'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.147350 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68288f4e2f57454fbb06eff0dd72cca3'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.156205 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68288f4e2f57454fbb06eff0dd72cca3'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.170133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-3674de3e-11d3-44fe-b5a3-6804c5dd8d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68288f4e2f57454fbb06eff0dd72cca3'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.188329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1338/2672] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:16 2022] DELETE /identity/v3/users/68288f4e2f57454fbb06eff0dd72cca3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:16.384972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.400271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b514c-7644-4407-be82-bc8c4b6e7b8b service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.425854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:16.459155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.459798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.460091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.460358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] PATH_INFO: `/v3/projects/d255b03b31004c3b8e446f318cd72265` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.461455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d255b03b31004c3b8e446f318cd72265)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.463388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:16.513072 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd255b03b31004c3b8e446f318cd72265', 'request_id': 'req-a09482af-0564-472e-9f3e-b948bb324632'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.519412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd255b03b31004c3b8e446f318cd72265', 'request_id': 'req-a09482af-0564-472e-9f3e-b948bb324632'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.525022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd255b03b31004c3b8e446f318cd72265', 'request_id': 'req-a09482af-0564-472e-9f3e-b948bb324632'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.538623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] The token cache is being invalidate because project d255b03b31004c3b8e446f318cd72265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:16.538908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.539527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a09482af-0564-472e-9f3e-b948bb324632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:16.541026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1335/2673] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:16 2022] DELETE /identity/v3/projects/d255b03b31004c3b8e446f318cd72265 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:16.546067 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968506a3-362f-46a8-a7f7-b451dddd701d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.546378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968506a3-362f-46a8-a7f7-b451dddd701d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.546648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968506a3-362f-46a8-a7f7-b451dddd701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.566855 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-968506a3-362f-46a8-a7f7-b451dddd701d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:16.568131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-968506a3-362f-46a8-a7f7-b451dddd701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:16.630228 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1339/2674] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:16.639212 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa88f43-307a-424a-8b38-063f7a44318b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.639525 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa88f43-307a-424a-8b38-063f7a44318b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.639795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa88f43-307a-424a-8b38-063f7a44318b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.663035 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-7fa88f43-307a-424a-8b38-063f7a44318b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:16.664855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7fa88f43-307a-424a-8b38-063f7a44318b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:16.675305 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1336/2675] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:16.681268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968506a3-362f-46a8-a7f7-b451dddd701d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.691738 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968506a3-362f-46a8-a7f7-b451dddd701d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.697841 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.698438 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.698711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.698976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.699647 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:16.699977 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.701568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:16.714160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1340/2676] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 09:03:16.721109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa88f43-307a-424a-8b38-063f7a44318b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.735384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa88f43-307a-424a-8b38-063f7a44318b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.741957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.743213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.743615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.744045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.744999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.748029 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:16.778709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1337/2677] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:16 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:16.783188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.793800 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f686d-792b-4132-9c70-09c9aaf1ba23 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.847018 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:16.857640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.859485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.859485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.859485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.860398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.863290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:16.888500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1341/2678] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:16 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:03:16.892850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.903159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aedbb1-39e3-4a96-bd38-670731a2f3de admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.907999 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.908567 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.908840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.909103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.909748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.911334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:16.922148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1338/2679] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:16.926419 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.936910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f7a2-36c9-4cd0-b7a1-dba296ef1edd admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.941856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027bb4c-dfed-44de-a974-8a418923a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.942477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027bb4c-dfed-44de-a974-8a418923a96a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.942753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027bb4c-dfed-44de-a974-8a418923a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.943024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027bb4c-dfed-44de-a974-8a418923a96a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.943670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027bb4c-dfed-44de-a974-8a418923a96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.945302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027bb4c-dfed-44de-a974-8a418923a96a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:16.956869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1342/2680] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:03:16.961218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:16.971991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1a953-97fb-4b6b-8779-2987ea49e65f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:16.976998 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:16.977649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:16.977923 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:16.978178 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] PATH_INFO: `/v3/projects/152b5480c09c41a1a98b34eb90578fd6/users/ecb51d6d3bf74bc78f47b42e355b7d98/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:16.995352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=152b5480c09c41a1a98b34eb90578fd6, user_id=ecb51d6d3bf74bc78f47b42e355b7d98, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:16.998131 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.015868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1339/2681] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:16 2022] PUT /identity/v3/projects/152b5480c09c41a1a98b34eb90578fd6/users/ecb51d6d3bf74bc78f47b42e355b7d98/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:17.031684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292b16-ba1b-42d8-94f1-8235293d836e None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.032014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292b16-ba1b-42d8-94f1-8235293d836e None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.032296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292b16-ba1b-42d8-94f1-8235293d836e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.061513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-48292b16-ba1b-42d8-94f1-8235293d836e None None] MFA Rules not processed for user `ecb51d6d3bf74bc78f47b42e355b7d98`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:17.062726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-48292b16-ba1b-42d8-94f1-8235293d836e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:17.131636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1343/2682] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:17.136440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:17.191228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:17.202394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f1a75-8f80-4e3e-af0e-66dae590c3d9 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:17.208334 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:17.208940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.209215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.209488 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.210135 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:17.212057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.241091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1340/2683] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:17 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:17.245621 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292b16-ba1b-42d8-94f1-8235293d836e None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:17.256830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292b16-ba1b-42d8-94f1-8235293d836e None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:17.316573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:17.317389 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.317761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.318152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.318890 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:17.321271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.347448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1344/2684] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:17 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:17.348967 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:17.361059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0a07c-5f16-4aef-9981-fad946b6a60f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:17.365968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:17.366690 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.366969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.367580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.368357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:17.369978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.380780 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1341/2685] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:17.385035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:17.395096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6a538-fadb-4280-805d-983e625561df admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:17.399698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:17.400391 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.400666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.400936 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] PATH_INFO: `/v3/projects/b06237a7394f488d9bf840f5f3e5e374/users/d3af41439f4142a88336a18427a9ff0f/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.417027 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06237a7394f488d9bf840f5f3e5e374, user_id=d3af41439f4142a88336a18427a9ff0f, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:17.419645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.437152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1345/2686] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:17 2022] PUT /identity/v3/projects/b06237a7394f488d9bf840f5f3e5e374/users/d3af41439f4142a88336a18427a9ff0f/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:17.441368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:17.451409 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ae0d-c779-437b-b016-091d04c62c81 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:17.512145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ba86-75e6-443f-a1d5-2a2fe5de5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:17.512771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ba86-75e6-443f-a1d5-2a2fe5de5814 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.513051 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ba86-75e6-443f-a1d5-2a2fe5de5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.513308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ba86-75e6-443f-a1d5-2a2fe5de5814 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.513952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7ba86-75e6-443f-a1d5-2a2fe5de5814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:17.515562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7ba86-75e6-443f-a1d5-2a2fe5de5814 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.525668 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1342/2687] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:17.531745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:17.542550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a7ceb-64b2-44d1-a5d8-e6d80b574898 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:17.547458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:17.548141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.548425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.548703 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] PATH_INFO: `/v3/projects/b06237a7394f488d9bf840f5f3e5e374/users/d3af41439f4142a88336a18427a9ff0f/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.550684 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06237a7394f488d9bf840f5f3e5e374, user_id=d3af41439f4142a88336a18427a9ff0f, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:17.553372 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.565210 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1346/2688] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:17 2022] PUT /identity/v3/projects/b06237a7394f488d9bf840f5f3e5e374/users/d3af41439f4142a88336a18427a9ff0f/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:17.575704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da843fc1-b406-4fbe-a975-c25767ddcd53 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.576025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da843fc1-b406-4fbe-a975-c25767ddcd53 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.576293 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da843fc1-b406-4fbe-a975-c25767ddcd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.606354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-da843fc1-b406-4fbe-a975-c25767ddcd53 None None] MFA Rules not processed for user `d3af41439f4142a88336a18427a9ff0f`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:17.607580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-da843fc1-b406-4fbe-a975-c25767ddcd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:17.677164 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1343/2689] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:17.688735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:17.690603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:17.711254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:17.754309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbeaac-0a1e-49c9-be55-14d226e7da88 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:17.758477 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2048d-424c-430b-a19e-eb678a8ca366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:17.759082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2048d-424c-430b-a19e-eb678a8ca366 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:17.759360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2048d-424c-430b-a19e-eb678a8ca366 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:17.759629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2048d-424c-430b-a19e-eb678a8ca366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:17.769013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2048d-424c-430b-a19e-eb678a8ca366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:17.770443 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2048d-424c-430b-a19e-eb678a8ca366 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:17.779967 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1347/2690] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:18.198980 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da843fc1-b406-4fbe-a975-c25767ddcd53 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:18.247859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:18.262326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da843fc1-b406-4fbe-a975-c25767ddcd53 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:18.267483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:18.268149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:18.268422 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:18.268702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] PATH_INFO: `/v3/users/ecb51d6d3bf74bc78f47b42e355b7d98` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:18.269815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb51d6d3bf74bc78f47b42e355b7d98)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:18.271771 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:18.320169 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb51d6d3bf74bc78f47b42e355b7d98'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:18.320827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb51d6d3bf74bc78f47b42e355b7d98'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:18.328101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb51d6d3bf74bc78f47b42e355b7d98'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:18.328773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb51d6d3bf74bc78f47b42e355b7d98'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:18.341256 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb51d6d3bf74bc78f47b42e355b7d98'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:18.356992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1344/2691] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:18 2022] DELETE /identity/v3/users/ecb51d6d3bf74bc78f47b42e355b7d98 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:18.361868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:18.362352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:18.362450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:18.381886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:18.383080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:18.450865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1348/2692] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:18.488125 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:18.489623 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:18.552129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c258f9-523a-4f02-a8f7-8bc84372fbec admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:18.556791 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:18.557714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:18.557714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:18.557986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:18.568612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:18.570050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:18.580663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1345/2693] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 09:03:19.281266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:19.282935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:19.296495 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e62ce3-11df-4ba2-8c2e-e9235733bd94 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:19.303706 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:19.304337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:19.304598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:19.304869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] PATH_INFO: `/v3/projects/152b5480c09c41a1a98b34eb90578fd6` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:19.306124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=152b5480c09c41a1a98b34eb90578fd6)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:19.308133 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:19.360454 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152b5480c09c41a1a98b34eb90578fd6', 'request_id': 'req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.367830 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152b5480c09c41a1a98b34eb90578fd6', 'request_id': 'req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.376511 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152b5480c09c41a1a98b34eb90578fd6', 'request_id': 'req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.393446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] The token cache is being invalidate because project 152b5480c09c41a1a98b34eb90578fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:19.393730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.394356 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.396012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1349/2694] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:19 2022] DELETE /identity/v3/projects/152b5480c09c41a1a98b34eb90578fd6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:19.400129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:19.409882 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d401810-a1d1-4b26-84d2-2fdeaf4626a8 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:19.467863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:19.468512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:19.468783 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:19.469042 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] PATH_INFO: `/v3/users/d3af41439f4142a88336a18427a9ff0f` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:19.470145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3af41439f4142a88336a18427a9ff0f)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:19.472101 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:19.485062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:19.521836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3af41439f4142a88336a18427a9ff0f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.522502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3af41439f4142a88336a18427a9ff0f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.529284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3af41439f4142a88336a18427a9ff0f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.530011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3af41439f4142a88336a18427a9ff0f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.541622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9f9d15eb-ac13-4939-8fcb-1873c95e27b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3af41439f4142a88336a18427a9ff0f'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:19.555533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1346/2695] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:19 2022] DELETE /identity/v3/users/d3af41439f4142a88336a18427a9ff0f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:19.875737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:19.877431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:19.892005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27dd9d-27db-4286-a4ba-8b76d0bec0fe admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:19.938665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:19.963450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:19.964222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:19.964561 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:19.964833 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] PATH_INFO: `/v3/projects/b06237a7394f488d9bf840f5f3e5e374` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:19.965958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06237a7394f488d9bf840f5f3e5e374)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:19.967927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.020160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06237a7394f488d9bf840f5f3e5e374', 'request_id': 'req-af19d117-bac8-49aa-855a-d2c672f51a34'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:20.026736 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06237a7394f488d9bf840f5f3e5e374', 'request_id': 'req-af19d117-bac8-49aa-855a-d2c672f51a34'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:20.032576 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06237a7394f488d9bf840f5f3e5e374', 'request_id': 'req-af19d117-bac8-49aa-855a-d2c672f51a34'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:20.046858 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] The token cache is being invalidate because project b06237a7394f488d9bf840f5f3e5e374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:20.047150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:20.047772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-af19d117-bac8-49aa-855a-d2c672f51a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:20.049324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1350/2696] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:19 2022] DELETE /identity/v3/projects/b06237a7394f488d9bf840f5f3e5e374 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:20.054606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9186-9046-4b31-bd5f-011a8b06ce44 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.054891 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9186-9046-4b31-bd5f-011a8b06ce44 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.055163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9186-9046-4b31-bd5f-011a8b06ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.077109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-48cf9186-9046-4b31-bd5f-011a8b06ce44 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:20.078269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-48cf9186-9046-4b31-bd5f-011a8b06ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:20.140262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1347/2697] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:20.148204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab95171-2445-45d0-b887-ee4597fc04cc None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.148493 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab95171-2445-45d0-b887-ee4597fc04cc None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.148764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab95171-2445-45d0-b887-ee4597fc04cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.168816 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5ab95171-2445-45d0-b887-ee4597fc04cc None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:20.170032 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5ab95171-2445-45d0-b887-ee4597fc04cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:20.178739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1351/2698] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:20.183117 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf9186-9046-4b31-bd5f-011a8b06ce44 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.193142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf9186-9046-4b31-bd5f-011a8b06ce44 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.198179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.198781 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.199057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.199326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.200009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:20.200313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.201927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.213600 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1348/2699] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:20.218328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab95171-2445-45d0-b887-ee4597fc04cc None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.228590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab95171-2445-45d0-b887-ee4597fc04cc None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.234743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.235347 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.235618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.235904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.236536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.238425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.272619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1352/2700] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:20 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:20.276904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.287188 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03793d-ada5-42fb-a406-1da985c42822 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.344798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.345327 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.345594 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.345859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.346478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.348402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.371711 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1349/2701] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:20 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:20.376288 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.386605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84241034-57e8-4478-ac16-f09f1a703784 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.391775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.392395 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.392666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.392931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.393597 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.395156 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.405974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1353/2702] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:20.409902 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.419880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6c4de-2715-40c8-81dd-d32a6f336020 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.425219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba93938-75c9-41d8-aa66-fac2381ab1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.426268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba93938-75c9-41d8-aa66-fac2381ab1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.426774 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba93938-75c9-41d8-aa66-fac2381ab1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.427109 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba93938-75c9-41d8-aa66-fac2381ab1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.427922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba93938-75c9-41d8-aa66-fac2381ab1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.429609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba93938-75c9-41d8-aa66-fac2381ab1f8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.431155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:20.442318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1350/2703] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:20.446375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.456755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a9ece-f21a-4bbf-9fa6-8d92748f160a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.462304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.462960 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.463239 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.463509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] PATH_INFO: `/v3/projects/637faeb50ec14a51aad455ccf9b59113/users/c0c2eb58a95542dc8276a0412c6f0332/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.479393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=637faeb50ec14a51aad455ccf9b59113, user_id=c0c2eb58a95542dc8276a0412c6f0332, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.482083 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.497739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1354/2704] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:20 2022] PUT /identity/v3/projects/637faeb50ec14a51aad455ccf9b59113/users/c0c2eb58a95542dc8276a0412c6f0332/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:20.508474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc13ba4-df6d-424d-acb2-2f0e98314277 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.508827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc13ba4-df6d-424d-acb2-2f0e98314277 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.509213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc13ba4-df6d-424d-acb2-2f0e98314277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.538417 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5dc13ba4-df6d-424d-acb2-2f0e98314277 None None] MFA Rules not processed for user `c0c2eb58a95542dc8276a0412c6f0332`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:20.539688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5dc13ba4-df6d-424d-acb2-2f0e98314277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:20.607746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1351/2705] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:20.612392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.678124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbea28-9ca1-4458-90d8-3ae69a617262 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.685143 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.686085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.686507 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.686980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.689003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.692114 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.710036 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:20.725987 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1355/2706] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:20 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:20.730205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc13ba4-df6d-424d-acb2-2f0e98314277 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.741520 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc13ba4-df6d-424d-acb2-2f0e98314277 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.800416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.801054 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.801326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.801599 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.802234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.804165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.830502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1352/2707] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:20 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:03:20.831705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.842412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864934a8-ec36-4787-912d-d0b5900e6a2e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.847131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.847737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.848053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.848327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.848973 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.850583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.864525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1356/2708] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:20.865735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.876538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8d735-ad70-4af3-8e25-2251aa54b767 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:20.881071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:20.881813 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:20.882058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:20.882332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] PATH_INFO: `/v3/projects/10d9253c2eb640ec8051921c4920e5b3/users/10ed24fd47ef452091015cc3f74e5afc/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:20.900480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d9253c2eb640ec8051921c4920e5b3, user_id=10ed24fd47ef452091015cc3f74e5afc, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:20.903243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:20.918501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1353/2709] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:20 2022] PUT /identity/v3/projects/10d9253c2eb640ec8051921c4920e5b3/users/10ed24fd47ef452091015cc3f74e5afc/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:20.922774 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:20.933344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a33a54-82be-4dfd-b636-ba8b64ddab85 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:21.009262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20169989-266b-43df-93b8-fd2bdc78f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:21.009907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20169989-266b-43df-93b8-fd2bdc78f147 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:21.010184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20169989-266b-43df-93b8-fd2bdc78f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:21.010457 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20169989-266b-43df-93b8-fd2bdc78f147 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:21.011102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20169989-266b-43df-93b8-fd2bdc78f147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:21.012766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20169989-266b-43df-93b8-fd2bdc78f147 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:21.022373 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1357/2710] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:21.026636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:21.037852 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab448a71-9549-4063-9022-adbc8dd6cccc admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:21.042663 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:21.043306 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:21.043579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:21.043841 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] PATH_INFO: `/v3/projects/10d9253c2eb640ec8051921c4920e5b3/users/10ed24fd47ef452091015cc3f74e5afc/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:21.046015 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d9253c2eb640ec8051921c4920e5b3, user_id=10ed24fd47ef452091015cc3f74e5afc, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:21.048505 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:21.061940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1354/2711] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:21 2022] PUT /identity/v3/projects/10d9253c2eb640ec8051921c4920e5b3/users/10ed24fd47ef452091015cc3f74e5afc/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:21.073268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e6ef-4c42-4ee7-84aa-e5841f160115 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:21.073545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e6ef-4c42-4ee7-84aa-e5841f160115 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:21.073861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e6ef-4c42-4ee7-84aa-e5841f160115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:21.104388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-d6c2e6ef-4c42-4ee7-84aa-e5841f160115 None None] MFA Rules not processed for user `10ed24fd47ef452091015cc3f74e5afc`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:21.106235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d6c2e6ef-4c42-4ee7-84aa-e5841f160115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:21.180627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1358/2712] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:21.191548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:21.193546 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:21.238478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:21.256613 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac8851-4df0-44bc-a951-e9eda1a81234 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:21.260746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f77af-0c67-40b3-b3f6-4f3b73d4d072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:21.261343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f77af-0c67-40b3-b3f6-4f3b73d4d072 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:21.261620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f77af-0c67-40b3-b3f6-4f3b73d4d072 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:21.261887 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f77af-0c67-40b3-b3f6-4f3b73d4d072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:21.271222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f77af-0c67-40b3-b3f6-4f3b73d4d072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:21.272687 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f77af-0c67-40b3-b3f6-4f3b73d4d072 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:21.282093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1355/2713] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:22.921750 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2e6ef-4c42-4ee7-84aa-e5841f160115 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:22.989877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2e6ef-4c42-4ee7-84aa-e5841f160115 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:22.995539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:22.996298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:23.000637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:23.000922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] PATH_INFO: `/v3/users/c0c2eb58a95542dc8276a0412c6f0332` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:23.002217 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c2eb58a95542dc8276a0412c6f0332)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:23.004187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:23.017172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:23.074500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c2eb58a95542dc8276a0412c6f0332'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.075228 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c2eb58a95542dc8276a0412c6f0332'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.075868 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c2eb58a95542dc8276a0412c6f0332'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.083146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c2eb58a95542dc8276a0412c6f0332'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.095367 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c2eb58a95542dc8276a0412c6f0332'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.110380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1359/2714] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:22 2022] DELETE /identity/v3/users/c0c2eb58a95542dc8276a0412c6f0332 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:23.115225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:23.115745 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:23.116106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:23.136989 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:23.138201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:23.197365 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1356/2715] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:23.207481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:23.209291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:23.271071 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98294cc8-4c75-4e70-9b37-1aef5fd725a8 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:23.274992 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:23.275606 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:23.276059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:23.276333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:23.287222 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:23.288978 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:23.298693 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1360/2716] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:23.569247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:23.570714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:23.585641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0b89-1096-41c8-9937-8c32e442cc7e None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:23.590524 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:23.591225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:23.591446 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:23.591749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] PATH_INFO: `/v3/projects/637faeb50ec14a51aad455ccf9b59113` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:23.592893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=637faeb50ec14a51aad455ccf9b59113)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:23.594987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:23.644754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637faeb50ec14a51aad455ccf9b59113', 'request_id': 'req-b083e303-ccb4-405b-a1bb-816f4956d134'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.650798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637faeb50ec14a51aad455ccf9b59113', 'request_id': 'req-b083e303-ccb4-405b-a1bb-816f4956d134'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.656968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637faeb50ec14a51aad455ccf9b59113', 'request_id': 'req-b083e303-ccb4-405b-a1bb-816f4956d134'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.672919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] The token cache is being invalidate because project 637faeb50ec14a51aad455ccf9b59113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:23.673280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.673925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.675440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1357/2717] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:23 2022] DELETE /identity/v3/projects/637faeb50ec14a51aad455ccf9b59113 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:23.679559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:23.698582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843188ad-e05c-4116-9367-9b714615e5aa service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:23.765362 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:23.766202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:23.766489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:23.766817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] PATH_INFO: `/v3/users/10ed24fd47ef452091015cc3f74e5afc` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:23.768041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ed24fd47ef452091015cc3f74e5afc)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:23.769918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:23.804354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:23.824657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ed24fd47ef452091015cc3f74e5afc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.825349 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ed24fd47ef452091015cc3f74e5afc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.825963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ed24fd47ef452091015cc3f74e5afc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.833203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ed24fd47ef452091015cc3f74e5afc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.845866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-647a141c-dfc4-4bd8-8742-7f27c7a21d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ed24fd47ef452091015cc3f74e5afc'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:23.861414 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1361/2718] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:23 2022] DELETE /identity/v3/users/10ed24fd47ef452091015cc3f74e5afc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:24.150106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.151597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:24.165776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083e303-ccb4-405b-a1bb-816f4956d134 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.223792 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.224448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.224723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.225011 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] PATH_INFO: `/v3/projects/10d9253c2eb640ec8051921c4920e5b3` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.226144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d9253c2eb640ec8051921c4920e5b3)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.228091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.234405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:24.280779 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d9253c2eb640ec8051921c4920e5b3', 'request_id': 'req-acf433c3-8bf9-4321-945a-318fb5dbc0cb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:24.286898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d9253c2eb640ec8051921c4920e5b3', 'request_id': 'req-acf433c3-8bf9-4321-945a-318fb5dbc0cb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:24.292620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d9253c2eb640ec8051921c4920e5b3', 'request_id': 'req-acf433c3-8bf9-4321-945a-318fb5dbc0cb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:24.306837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] The token cache is being invalidate because project 10d9253c2eb640ec8051921c4920e5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:24.307157 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:24.307846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-acf433c3-8bf9-4321-945a-318fb5dbc0cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:24.309378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1358/2719] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:24 2022] DELETE /identity/v3/projects/10d9253c2eb640ec8051921c4920e5b3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:24.314995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fcf94-2b7f-485a-8fa4-7c2de1177366 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.315485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fcf94-2b7f-485a-8fa4-7c2de1177366 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.315584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fcf94-2b7f-485a-8fa4-7c2de1177366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.336662 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-7e2fcf94-2b7f-485a-8fa4-7c2de1177366 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:24.338742 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-7e2fcf94-2b7f-485a-8fa4-7c2de1177366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:24.407077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1362/2720] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:24.415049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc8e7-6336-440c-bbd2-ab8ec77b4c6d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.415413 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc8e7-6336-440c-bbd2-ab8ec77b4c6d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.415764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc8e7-6336-440c-bbd2-ab8ec77b4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.437699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-071bc8e7-6336-440c-bbd2-ab8ec77b4c6d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:24.438994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-071bc8e7-6336-440c-bbd2-ab8ec77b4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:24.447728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1359/2721] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:24.452202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fcf94-2b7f-485a-8fa4-7c2de1177366 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.462687 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fcf94-2b7f-485a-8fa4-7c2de1177366 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.467296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.467922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.468236 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.468510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.469160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:24.469463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.471062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.483610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1363/2722] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:24.488173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bc8e7-6336-440c-bbd2-ab8ec77b4c6d None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.498195 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bc8e7-6336-440c-bbd2-ab8ec77b4c6d None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.503373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.504007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.504329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.504555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.505317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.507229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.536716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1360/2723] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:24 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:03:24.541344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.551980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e69299-d8f5-4948-bc79-35b451241b4e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.610375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.611100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.611377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.611657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.612325 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.614251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.637445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1364/2724] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:24 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:24.641646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.652310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b4ec-a917-4468-980b-92ea4a7cc614 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.657908 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.658550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.658830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.659098 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.659753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.661397 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.672174 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1361/2725] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:03:24.676199 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.686430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ad2bd-a0fa-4cab-a2fb-737d6680c082 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.692249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83a847-4cd0-40af-b3fb-457e65735e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.692363 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83a847-4cd0-40af-b3fb-457e65735e05 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.692608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83a847-4cd0-40af-b3fb-457e65735e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.692878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83a847-4cd0-40af-b3fb-457e65735e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.693532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83a847-4cd0-40af-b3fb-457e65735e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.695122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83a847-4cd0-40af-b3fb-457e65735e05 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.708594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1365/2726] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:24.712853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.723491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd2310-3ef4-4a91-a1e4-41d12c27b2bd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.728223 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.728873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.729148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.729557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] PATH_INFO: `/v3/projects/cd8dcb947ea94427a476ec3896639d28/users/2b48200620c245c5995856b6f6a71fde/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.745787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8dcb947ea94427a476ec3896639d28, user_id=2b48200620c245c5995856b6f6a71fde, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.748515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.764415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1362/2727] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:24 2022] PUT /identity/v3/projects/cd8dcb947ea94427a476ec3896639d28/users/2b48200620c245c5995856b6f6a71fde/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:24.775202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e201-3d5d-4a97-8b31-ff1d43fbcb00 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.775510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e201-3d5d-4a97-8b31-ff1d43fbcb00 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.775781 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e201-3d5d-4a97-8b31-ff1d43fbcb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.797343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:24.805594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-6216e201-3d5d-4a97-8b31-ff1d43fbcb00 None None] MFA Rules not processed for user `2b48200620c245c5995856b6f6a71fde`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:24.808471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-6216e201-3d5d-4a97-8b31-ff1d43fbcb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:24.875913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1366/2728] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:24.880638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:24.946751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266009f3-f078-48f3-9aaf-5bf8a6e737af admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:24.951583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:24.952231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:24.952532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:24.952807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:24.953449 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:24.955405 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:24.988894 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1363/2729] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:24 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:24.993500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e201-3d5d-4a97-8b31-ff1d43fbcb00 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.004927 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e201-3d5d-4a97-8b31-ff1d43fbcb00 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.062806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.063335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.063650 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.064073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.064719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.066648 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.092956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1367/2730] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:24 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:25.094260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.104903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7890fc8-a4d2-48ab-8cf8-e34cf25ffb6d admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.110115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.110800 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.111078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.111341 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.112021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.113633 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.125010 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1364/2731] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:25.129659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.140697 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85168e7-228f-4b10-bbc1-3eccfab0ca59 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.146773 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.146998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.147284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.147528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] PATH_INFO: `/v3/projects/e7943516279442899d6e5edc9c87f5c3/users/cc488ad307bb4d74a26b76d65e35e3bd/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.164566 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7943516279442899d6e5edc9c87f5c3, user_id=cc488ad307bb4d74a26b76d65e35e3bd, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.167229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.182017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1368/2732] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:25 2022] PUT /identity/v3/projects/e7943516279442899d6e5edc9c87f5c3/users/cc488ad307bb4d74a26b76d65e35e3bd/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:25.186244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.192934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:25.197111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c150d-eb77-4be7-9bb7-550149516405 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.257081 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c626b-83d5-46d2-b1cc-6ece8fec6db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.257696 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c626b-83d5-46d2-b1cc-6ece8fec6db8 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.257974 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c626b-83d5-46d2-b1cc-6ece8fec6db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.258241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c626b-83d5-46d2-b1cc-6ece8fec6db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.258909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c626b-83d5-46d2-b1cc-6ece8fec6db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.260550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c626b-83d5-46d2-b1cc-6ece8fec6db8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.270583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1365/2733] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:25.274944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.285826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d55b9-59d5-416f-82c9-e697e6f62054 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.290705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.291361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.291641 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.291910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] PATH_INFO: `/v3/projects/e7943516279442899d6e5edc9c87f5c3/users/cc488ad307bb4d74a26b76d65e35e3bd/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.294300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7943516279442899d6e5edc9c87f5c3, user_id=cc488ad307bb4d74a26b76d65e35e3bd, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.296981 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.308413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1369/2734] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:25 2022] PUT /identity/v3/projects/e7943516279442899d6e5edc9c87f5c3/users/cc488ad307bb4d74a26b76d65e35e3bd/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:25.320060 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea763b6-fbf4-4585-8760-c5d006ea3f37 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.320241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea763b6-fbf4-4585-8760-c5d006ea3f37 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.320540 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea763b6-fbf4-4585-8760-c5d006ea3f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.348937 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9ea763b6-fbf4-4585-8760-c5d006ea3f37 None None] MFA Rules not processed for user `cc488ad307bb4d74a26b76d65e35e3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:25.350152 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9ea763b6-fbf4-4585-8760-c5d006ea3f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:25.421284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1366/2735] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:25.433906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.435726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:25.496613 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f4e6-c4b8-4d7d-9f2a-8c9e5787b92f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.501014 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.501753 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.502024 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.502287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.511478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:25.513907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.515587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.527229 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1370/2736] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:25.894596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea763b6-fbf4-4585-8760-c5d006ea3f37 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.903741 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea763b6-fbf4-4585-8760-c5d006ea3f37 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.908869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc488ad307bb4d74a26b76d65e35e3bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7943516279442899d6e5edc9c87f5c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.909557 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.909919 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.910243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.911210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:25.911342 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.913346 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.927922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1367/2737] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:25.929895 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.939151 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0113f70-83d1-473e-bcfd-9a9ff053f520 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:25.943695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc488ad307bb4d74a26b76d65e35e3bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7943516279442899d6e5edc9c87f5c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:25.944343 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:25.944624 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:25.944896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:25.945530 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:25.947583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:25.982034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1371/2738] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:25 2022] POST /identity/v3/projects => generated 354 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:25.986519 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:25.999328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed8253-c2c8-47a3-b45f-afd4b17ba8d9 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:26.084237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc488ad307bb4d74a26b76d65e35e3bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7943516279442899d6e5edc9c87f5c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:26.084889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:26.085165 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:26.085479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:26.086139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:26.086438 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:26.088215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:26.106986 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1368/2739] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:26.145127 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:26.155871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf3ec3-7ff2-4dc6-8eb2-e8366042a0ab tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:26.161081 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc488ad307bb4d74a26b76d65e35e3bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7943516279442899d6e5edc9c87f5c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:26.161818 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:26.162167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:26.163000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] PATH_INFO: `/v3/projects/c077d20aa48e4c3f8fa45cf38aa065e5` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:26.164622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c077d20aa48e4c3f8fa45cf38aa065e5)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:26.166409 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:26.218910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c077d20aa48e4c3f8fa45cf38aa065e5', 'request_id': 'req-dd7a4e8c-f949-48b5-88b2-734c17141740'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.227206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c077d20aa48e4c3f8fa45cf38aa065e5', 'request_id': 'req-dd7a4e8c-f949-48b5-88b2-734c17141740'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.233563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c077d20aa48e4c3f8fa45cf38aa065e5', 'request_id': 'req-dd7a4e8c-f949-48b5-88b2-734c17141740'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.252991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] The token cache is being invalidate because project c077d20aa48e4c3f8fa45cf38aa065e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:26.253286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.253941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dd7a4e8c-f949-48b5-88b2-734c17141740 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.259405 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1372/2740] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:26 2022] DELETE /identity/v3/projects/c077d20aa48e4c3f8fa45cf38aa065e5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:26.474597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:26.476450 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:26.487023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=100 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:26.549707 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68cc5c-c8b5-4af8-893e-ee90a7301363 tempest-VolumeQuotaClassesTest-1427251511 tempest-VolumeQuotaClassesTest-1427251511-project-admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:26.603552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:26.604222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:26.604498 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:26.604766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] PATH_INFO: `/v3/users/2b48200620c245c5995856b6f6a71fde` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:26.605941 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b48200620c245c5995856b6f6a71fde)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:26.607851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:26.659088 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b48200620c245c5995856b6f6a71fde'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.660231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b48200620c245c5995856b6f6a71fde'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.667280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b48200620c245c5995856b6f6a71fde'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.667965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b48200620c245c5995856b6f6a71fde'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.680754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b48200620c245c5995856b6f6a71fde'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:26.695314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1369/2741] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:26 2022] DELETE /identity/v3/users/2b48200620c245c5995856b6f6a71fde => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:26.699724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:26.700266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:26.700370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:26.719637 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:26.721355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:26.783755 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1373/2742] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:26.793842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:26.795384 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:26.831290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:26.855357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561230b8-ab91-4227-8ddd-df50f6b307e2 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:26.858994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:26.859606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:26.859906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:26.860218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:26.869782 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:26.871279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:26.881897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1370/2743] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:27.169278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:27.170747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:27.182832 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83278ca-c924-47cc-87be-4706ec0c9e95 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:27.188335 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:27.189182 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:27.189574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:27.189850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] PATH_INFO: `/v3/projects/cd8dcb947ea94427a476ec3896639d28` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:27.190988 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8dcb947ea94427a476ec3896639d28)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:27.192956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:27.246956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8dcb947ea94427a476ec3896639d28', 'request_id': 'req-d304130c-20a6-43dc-935e-6cbaa0939f16'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.249536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:27.254872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8dcb947ea94427a476ec3896639d28', 'request_id': 'req-d304130c-20a6-43dc-935e-6cbaa0939f16'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.261660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8dcb947ea94427a476ec3896639d28', 'request_id': 'req-d304130c-20a6-43dc-935e-6cbaa0939f16'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.276487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] The token cache is being invalidate because project cd8dcb947ea94427a476ec3896639d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:27.276775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.277407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.278917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1374/2744] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:27 2022] DELETE /identity/v3/projects/cd8dcb947ea94427a476ec3896639d28 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:27.283145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:27.292829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422bbca-fadf-4393-9d08-c29508d4171f service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:27.347988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:27.348609 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:27.348883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:27.349153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] PATH_INFO: `/v3/users/cc488ad307bb4d74a26b76d65e35e3bd` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:27.350276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc488ad307bb4d74a26b76d65e35e3bd)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:27.352239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:27.400447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc488ad307bb4d74a26b76d65e35e3bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.401024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc488ad307bb4d74a26b76d65e35e3bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.409389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc488ad307bb4d74a26b76d65e35e3bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.410057 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc488ad307bb4d74a26b76d65e35e3bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.426650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-cd5fe2b2-5d6a-48ae-ab21-4fff9aa793f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc488ad307bb4d74a26b76d65e35e3bd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.446766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1371/2745] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:27 2022] DELETE /identity/v3/users/cc488ad307bb4d74a26b76d65e35e3bd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:27.726327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:27.727845 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:27.752504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304130c-20a6-43dc-935e-6cbaa0939f16 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:27.812066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:27.812699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:27.812976 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:27.813262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] PATH_INFO: `/v3/projects/e7943516279442899d6e5edc9c87f5c3` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:27.814594 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7943516279442899d6e5edc9c87f5c3)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:27.816605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:27.867588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7943516279442899d6e5edc9c87f5c3', 'request_id': 'req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.874079 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7943516279442899d6e5edc9c87f5c3', 'request_id': 'req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.880172 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7943516279442899d6e5edc9c87f5c3', 'request_id': 'req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.895842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] The token cache is being invalidate because project e7943516279442899d6e5edc9c87f5c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:27.896173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.896820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-dc6b0e76-5b9e-4ddb-8ee9-a0cd63e8445b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:27.898351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1375/2746] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:27 2022] DELETE /identity/v3/projects/e7943516279442899d6e5edc9c87f5c3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:27.903354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721c31-4e10-44ba-a620-e3ee46d3f553 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:27.903839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721c31-4e10-44ba-a620-e3ee46d3f553 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:27.903979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721c31-4e10-44ba-a620-e3ee46d3f553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:27.924922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-92721c31-4e10-44ba-a620-e3ee46d3f553 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:27.926043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-92721c31-4e10-44ba-a620-e3ee46d3f553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:27.976667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:27.989579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1372/2747] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:27.997968 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3407d-a694-4be6-b685-52847eeb4e59 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:27.998250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3407d-a694-4be6-b685-52847eeb4e59 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:27.998523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3407d-a694-4be6-b685-52847eeb4e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.019730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e3c3407d-a694-4be6-b685-52847eeb4e59 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:28.020994 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e3c3407d-a694-4be6-b685-52847eeb4e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:28.029882 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1376/2748] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:28.034649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721c31-4e10-44ba-a620-e3ee46d3f553 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.045037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721c31-4e10-44ba-a620-e3ee46d3f553 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.050886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.051510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.051864 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.052161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.052883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:28.053234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.054922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.067470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1373/2749] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:28.071931 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3407d-a694-4be6-b685-52847eeb4e59 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.081629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3407d-a694-4be6-b685-52847eeb4e59 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.087073 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.087673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.087986 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.088268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.088904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.090778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.121059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1377/2750] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:28 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:03:28.125362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.136737 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbdec0-2c2a-4976-a95e-52fdd28356e7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.194311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.194833 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.195107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.195377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.196028 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.197934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.221448 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1374/2751] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:28 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:28.225896 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.236591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cdd11-ccad-435b-8a37-c7d8ee04709a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.241445 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.242371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.242658 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.242926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.243587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.245231 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.255284 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1378/2752] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:28.259348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.269818 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba33050-fcca-4ff1-bab3-d05a3e5def51 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.275215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c9677-a0fe-44a0-968e-ef8e28c8c78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.275812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c9677-a0fe-44a0-968e-ef8e28c8c78d admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.276124 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c9677-a0fe-44a0-968e-ef8e28c8c78d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.276390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c9677-a0fe-44a0-968e-ef8e28c8c78d admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.277027 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c9677-a0fe-44a0-968e-ef8e28c8c78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.278617 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c9677-a0fe-44a0-968e-ef8e28c8c78d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.289916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1375/2753] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:28.294326 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.305005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101a803-206a-4737-89bd-b3fa80277cb6 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.310396 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.311040 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.311312 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.311577 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] PATH_INFO: `/v3/projects/a7cb105eb086432684c7269e52a82a52/users/6a0c0e1e633644bd82c6ffa698732c36/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.326678 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7cb105eb086432684c7269e52a82a52, user_id=6a0c0e1e633644bd82c6ffa698732c36, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.329401 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.345001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1379/2754] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:28 2022] PUT /identity/v3/projects/a7cb105eb086432684c7269e52a82a52/users/6a0c0e1e633644bd82c6ffa698732c36/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:28.356698 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9c14b-e7dd-4fb8-9276-0fe8c2cec69c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.356993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9c14b-e7dd-4fb8-9276-0fe8c2cec69c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.357265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9c14b-e7dd-4fb8-9276-0fe8c2cec69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.386669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a6d9c14b-e7dd-4fb8-9276-0fe8c2cec69c None None] MFA Rules not processed for user `6a0c0e1e633644bd82c6ffa698732c36`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:28.387870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a6d9c14b-e7dd-4fb8-9276-0fe8c2cec69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:28.456956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1376/2755] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:28.461544 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.470759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:28.526106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e443992-36c7-47f6-b156-45994bf98393 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.531285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.531904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.532203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.532476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.533109 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.535064 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.568267 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1380/2756] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:28 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:28.569812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9c14b-e7dd-4fb8-9276-0fe8c2cec69c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.580938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9c14b-e7dd-4fb8-9276-0fe8c2cec69c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.629840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:28.642145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.642759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.643091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.643358 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.644013 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.645931 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.668677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1377/2757] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:28 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:28.672849 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.683235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fc9f2-8d05-4804-89b7-c111aac9d754 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.687999 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.688589 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.688870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.689137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.689777 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.691404 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.702877 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1381/2758] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:28.707127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.718146 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5a79-6dc5-496b-8e3b-64573bf980ba admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.724045 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.724685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.724956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.725219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] PATH_INFO: `/v3/projects/bd160996363c47128660fa215cea1f70/users/b70db280255a4ffd9bfbb2fc526a29b1/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.741140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd160996363c47128660fa215cea1f70, user_id=b70db280255a4ffd9bfbb2fc526a29b1, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.744667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.759099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1378/2759] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:28 2022] PUT /identity/v3/projects/bd160996363c47128660fa215cea1f70/users/b70db280255a4ffd9bfbb2fc526a29b1/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:28.763248 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.773844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b4270-e3f2-4a99-969e-730293ad4148 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.834466 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d176c-10f5-4563-91d8-9afb2dd1ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.835065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d176c-10f5-4563-91d8-9afb2dd1ff20 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.835341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d176c-10f5-4563-91d8-9afb2dd1ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.835608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d176c-10f5-4563-91d8-9afb2dd1ff20 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.836358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d176c-10f5-4563-91d8-9afb2dd1ff20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.837934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d176c-10f5-4563-91d8-9afb2dd1ff20 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.847542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1382/2760] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:28.853392 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:28.864245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071a07c-d470-422c-b7e8-157c8ba2fb15 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:28.869691 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:28.870339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.870620 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.870889 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] PATH_INFO: `/v3/projects/bd160996363c47128660fa215cea1f70/users/b70db280255a4ffd9bfbb2fc526a29b1/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.872904 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd160996363c47128660fa215cea1f70, user_id=b70db280255a4ffd9bfbb2fc526a29b1, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:28.875548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:28.885912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1379/2761] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:28 2022] PUT /identity/v3/projects/bd160996363c47128660fa215cea1f70/users/b70db280255a4ffd9bfbb2fc526a29b1/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:28.896289 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e76c7-4eb3-4c1e-bb4b-72f8eeaa5df5 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:28.896584 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e76c7-4eb3-4c1e-bb4b-72f8eeaa5df5 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:28.896857 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e76c7-4eb3-4c1e-bb4b-72f8eeaa5df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:28.929903 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-db0e76c7-4eb3-4c1e-bb4b-72f8eeaa5df5 None None] MFA Rules not processed for user `b70db280255a4ffd9bfbb2fc526a29b1`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:28.932526 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-db0e76c7-4eb3-4c1e-bb4b-72f8eeaa5df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:29.003654 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1383/2762] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:29.015985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:29.017987 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:29.075553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c022a-0fe4-432e-90c4-0d9a8222143b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:29.079129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:29.079732 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:29.080041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:29.080318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:29.089518 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:29.090917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:29.100506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1380/2763] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:29.191281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e76c7-4eb3-4c1e-bb4b-72f8eeaa5df5 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:29.196232 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:29.203013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e76c7-4eb3-4c1e-bb4b-72f8eeaa5df5 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:29.210292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3f0e3-b13a-4ea8-a069-f1ad0d1feb0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:29.211194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f0e3-b13a-4ea8-a069-f1ad0d1feb0f service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:29.211581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f0e3-b13a-4ea8-a069-f1ad0d1feb0f service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:29.211997 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f0e3-b13a-4ea8-a069-f1ad0d1feb0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:29.272297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3f0e3-b13a-4ea8-a069-f1ad0d1feb0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:29.273633 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3f0e3-b13a-4ea8-a069-f1ad0d1feb0f service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:29.282381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1384/2764] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:34.118743 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:34.193086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2a988-0287-4592-81a7-d7b7c053335f service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:34.197976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:34.198605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:34.198886 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:34.199160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] PATH_INFO: `/v3/users/6a0c0e1e633644bd82c6ffa698732c36` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:34.200362 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a0c0e1e633644bd82c6ffa698732c36)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:34.202291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:34.253077 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a0c0e1e633644bd82c6ffa698732c36'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.253764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a0c0e1e633644bd82c6ffa698732c36'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.255033 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:34.262062 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a0c0e1e633644bd82c6ffa698732c36'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.263144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a0c0e1e633644bd82c6ffa698732c36'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.275295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a0c0e1e633644bd82c6ffa698732c36'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.291319 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1381/2765] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:34 2022] DELETE /identity/v3/users/6a0c0e1e633644bd82c6ffa698732c36 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:34.295969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:34.296474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:34.296587 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:34.316835 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:34.318049 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:34.381428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1385/2766] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:34.391310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:34.392854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:34.463867 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414da54-e60f-483c-a0b4-4bda7bb27815 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:34.468147 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:34.468760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:34.469032 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:34.469297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:34.482233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:34.483661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:34.493666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1382/2767] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:34.780680 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:34.782337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:34.797351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdc427-e70b-470c-a0ff-d9cbce901f5f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:34.802283 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:34.803063 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:34.803342 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:34.803617 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] PATH_INFO: `/v3/projects/a7cb105eb086432684c7269e52a82a52` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:34.804847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7cb105eb086432684c7269e52a82a52)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:34.806775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:34.855579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7cb105eb086432684c7269e52a82a52', 'request_id': 'req-238c9440-431e-4ea4-bd37-51ad7f125a9e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.861768 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7cb105eb086432684c7269e52a82a52', 'request_id': 'req-238c9440-431e-4ea4-bd37-51ad7f125a9e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.868020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7cb105eb086432684c7269e52a82a52', 'request_id': 'req-238c9440-431e-4ea4-bd37-51ad7f125a9e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.882672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] The token cache is being invalidate because project a7cb105eb086432684c7269e52a82a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:34.882899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.883509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:34.885191 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1386/2768] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:34 2022] DELETE /identity/v3/projects/a7cb105eb086432684c7269e52a82a52 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:34.889494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:34.899684 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01670e54-00e7-4063-a536-810f524e9f89 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:34.963149 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:34.963812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:34.964099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:34.964371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] PATH_INFO: `/v3/users/b70db280255a4ffd9bfbb2fc526a29b1` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:34.965506 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70db280255a4ffd9bfbb2fc526a29b1)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:34.967419 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.015730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70db280255a4ffd9bfbb2fc526a29b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.016412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70db280255a4ffd9bfbb2fc526a29b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.023139 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70db280255a4ffd9bfbb2fc526a29b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.023744 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70db280255a4ffd9bfbb2fc526a29b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.033721 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:35.036059 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-5adc07b1-4ac5-49cc-9009-db7f72f22966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70db280255a4ffd9bfbb2fc526a29b1'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.051368 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1383/2769] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:34 2022] DELETE /identity/v3/users/b70db280255a4ffd9bfbb2fc526a29b1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:35.342286 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:35.344406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:35.358041 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c9440-431e-4ea4-bd37-51ad7f125a9e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:35.379488 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:35.416298 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:35.416846 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.417124 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.417394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] PATH_INFO: `/v3/projects/bd160996363c47128660fa215cea1f70` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.418501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd160996363c47128660fa215cea1f70)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:35.420432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.466627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd160996363c47128660fa215cea1f70', 'request_id': 'req-1ff1fcaf-f498-4857-9d63-74947e14f4ac'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.472769 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd160996363c47128660fa215cea1f70', 'request_id': 'req-1ff1fcaf-f498-4857-9d63-74947e14f4ac'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.478483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd160996363c47128660fa215cea1f70', 'request_id': 'req-1ff1fcaf-f498-4857-9d63-74947e14f4ac'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.493300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] The token cache is being invalidate because project bd160996363c47128660fa215cea1f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:35.493586 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.494275 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1ff1fcaf-f498-4857-9d63-74947e14f4ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:35.495861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1387/2770] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:35 2022] DELETE /identity/v3/projects/bd160996363c47128660fa215cea1f70 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:35.500848 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66053e3d-b47c-48d4-b3ec-06572d8585c3 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.501307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66053e3d-b47c-48d4-b3ec-06572d8585c3 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.501401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66053e3d-b47c-48d4-b3ec-06572d8585c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.521591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-66053e3d-b47c-48d4-b3ec-06572d8585c3 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:35.522797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-66053e3d-b47c-48d4-b3ec-06572d8585c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:35.585982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1384/2771] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:35.594230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc1b3-ad0a-4235-a7d2-8ed9a99ed4f2 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.594487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc1b3-ad0a-4235-a7d2-8ed9a99ed4f2 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.594785 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc1b3-ad0a-4235-a7d2-8ed9a99ed4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.614886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4b5bc1b3-ad0a-4235-a7d2-8ed9a99ed4f2 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:35.616099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4b5bc1b3-ad0a-4235-a7d2-8ed9a99ed4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:35.625061 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1388/2772] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:35.629601 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66053e3d-b47c-48d4-b3ec-06572d8585c3 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:35.640350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66053e3d-b47c-48d4-b3ec-06572d8585c3 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:35.646009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:35.646607 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.646880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.647142 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.647784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:35.648114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:35.649702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.661922 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1385/2773] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:35.666503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bc1b3-ad0a-4235-a7d2-8ed9a99ed4f2 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:35.676281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bc1b3-ad0a-4235-a7d2-8ed9a99ed4f2 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:35.681935 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:35.682559 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.682840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.683130 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.683762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:35.685698 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.715378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1389/2774] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:35 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:35.719555 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:35.729858 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422bb92-c34a-4a45-be67-8ace959c216f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:35.790576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:35.791182 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.791455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.791723 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.792373 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:35.794286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.817231 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1386/2775] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:35 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:35.821465 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:35.832307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584e72-0c63-4bc5-acda-f44857c7d6df admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:35.838005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:35.838715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.838998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.839269 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.839923 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:35.841545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.852306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1390/2776] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:35.856272 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:35.866583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdd407-7946-4ab7-ae39-828b86a6f4be admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:35.871796 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a04e2-8f26-4417-9473-2528c428cb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:35.872432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a04e2-8f26-4417-9473-2528c428cb80 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.872712 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a04e2-8f26-4417-9473-2528c428cb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.872984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a04e2-8f26-4417-9473-2528c428cb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.873641 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a04e2-8f26-4417-9473-2528c428cb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:35.875233 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a04e2-8f26-4417-9473-2528c428cb80 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.886382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1387/2777] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:03:35.890503 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:35.901101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7ef67-a672-4fa2-be80-3feaf600b318 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:35.906499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:35.907148 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.907424 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.907692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] PATH_INFO: `/v3/projects/5dc4ed583f574cf8a12535b79d14021b/users/f67cbfb87712434ab48b8e8a3cb44994/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.924080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc4ed583f574cf8a12535b79d14021b, user_id=f67cbfb87712434ab48b8e8a3cb44994, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:35.926611 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:35.941509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1391/2778] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:35 2022] PUT /identity/v3/projects/5dc4ed583f574cf8a12535b79d14021b/users/f67cbfb87712434ab48b8e8a3cb44994/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:35.951964 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e68fd-31cc-45a7-8b25-92cda0b2bf5d None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:35.952388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e68fd-31cc-45a7-8b25-92cda0b2bf5d None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:35.952846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e68fd-31cc-45a7-8b25-92cda0b2bf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:35.979809 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2e9e68fd-31cc-45a7-8b25-92cda0b2bf5d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:35.981048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2e9e68fd-31cc-45a7-8b25-92cda0b2bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:36.031402 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:36.048832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1388/2779] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:36.062900 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:36.065186 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:36.126153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f30b18-5eb4-4cf5-97c8-abbd5bc93b52 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:36.130450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:36.131048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:36.131329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:36.131582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:36.137966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:36.141812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:36.143645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:36.153809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1392/2780] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:36.843371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247104a-9c09-45d0-981c-70f2f940786f None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:36.844002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247104a-9c09-45d0-981c-70f2f940786f None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:36.844002 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247104a-9c09-45d0-981c-70f2f940786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:36.873541 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-9247104a-9c09-45d0-981c-70f2f940786f None None] MFA Rules not processed for user `f67cbfb87712434ab48b8e8a3cb44994`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:36.874768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-9247104a-9c09-45d0-981c-70f2f940786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:36.938878 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1389/2781] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:36.943937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:36.954203 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833c039-ae0a-472d-9f6b-783151d34722 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:36.958899 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:36.959533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:36.959815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:36.960115 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:36.960756 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:36.962642 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:36.993105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1393/2782] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:36 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:36.997292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247104a-9c09-45d0-981c-70f2f940786f None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:37.008046 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247104a-9c09-45d0-981c-70f2f940786f None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:37.064839 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:37.065490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:37.065759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:37.066024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:37.066664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:37.068670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:37.092553 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1390/2783] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:36 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:37.096638 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:37.107066 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d782-afd8-4a42-8061-210a31b2c7bc admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:37.112184 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:37.112783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:37.113056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:37.113319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:37.113955 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:37.115536 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:37.126803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1394/2784] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:03:37.131069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:37.141676 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe56eed-a1f8-4950-a4c4-a736d06593a6 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:37.148185 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:37.148899 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:37.149180 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:37.149452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] PATH_INFO: `/v3/projects/93a119c411ed4796892b829a02ccf43d/users/2a67185bb2ec4c2681ec64890cb0ed3e/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:37.165270 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a119c411ed4796892b829a02ccf43d, user_id=2a67185bb2ec4c2681ec64890cb0ed3e, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:37.167965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:37.182280 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1391/2785] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:37 2022] PUT /identity/v3/projects/93a119c411ed4796892b829a02ccf43d/users/2a67185bb2ec4c2681ec64890cb0ed3e/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:37.186215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:37.196512 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89f36f-a810-44d9-a58c-e1e8fc894d4c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:37.255619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ecfea-8e40-4cee-a803-e329daef8ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:37.256257 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ecfea-8e40-4cee-a803-e329daef8ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:37.256527 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ecfea-8e40-4cee-a803-e329daef8ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:37.256803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ecfea-8e40-4cee-a803-e329daef8ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:37.257434 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ecfea-8e40-4cee-a803-e329daef8ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:37.259015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ecfea-8e40-4cee-a803-e329daef8ac5 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:37.269740 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1395/2786] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:37.274122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:37.285115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f4906-e118-4fe4-88d9-ea6484b4af82 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:37.290292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:37.290949 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:37.291222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:37.291494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] PATH_INFO: `/v3/projects/93a119c411ed4796892b829a02ccf43d/users/2a67185bb2ec4c2681ec64890cb0ed3e/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:37.293580 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a119c411ed4796892b829a02ccf43d, user_id=2a67185bb2ec4c2681ec64890cb0ed3e, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:37.296286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:37.306740 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1392/2787] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:37 2022] PUT /identity/v3/projects/93a119c411ed4796892b829a02ccf43d/users/2a67185bb2ec4c2681ec64890cb0ed3e/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:37.935020 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:37.935308 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:37.935629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:37.967922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] MFA Rules not processed for user `2a67185bb2ec4c2681ec64890cb0ed3e`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:37.969303 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:38.038194 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1396/2788] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:38.049708 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:38.051534 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:38.093503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:38.111605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f39d6-573c-4d4d-8a1d-cf516a2accf7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:38.115279 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:38.115907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:38.116187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:38.116452 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:38.172248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:38.173679 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:38.183116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1393/2789] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:39.591154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:39.592834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:39.606640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ed4f7-da14-485f-b73a-62b4af38e69f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:39.611270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:39.611909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:39.612196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:39.612468 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:39.623230 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:39.624694 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:39.634365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1397/2790] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:48.178874 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=32 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:49.630436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:52.927954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:52.992742 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3029c9f-5110-4bf5-a113-d9ebf4b1c000 service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:52.997348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:52.997978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:52.998257 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:52.998523 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] PATH_INFO: `/v3/users/f67cbfb87712434ab48b8e8a3cb44994` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:52.999757 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67cbfb87712434ab48b8e8a3cb44994)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:53.001699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:53.053322 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67cbfb87712434ab48b8e8a3cb44994'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.054007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67cbfb87712434ab48b8e8a3cb44994'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.060861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67cbfb87712434ab48b8e8a3cb44994'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.061485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67cbfb87712434ab48b8e8a3cb44994'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.074143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67cbfb87712434ab48b8e8a3cb44994'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.088795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1394/2791] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:52 2022] DELETE /identity/v3/users/f67cbfb87712434ab48b8e8a3cb44994 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:53.266704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:53.277861 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b62ee9-c78a-46cc-99be-3e8f0de2d4fb service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:53.336803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:53.337428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:53.337688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:53.337956 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] PATH_INFO: `/v3/projects/5dc4ed583f574cf8a12535b79d14021b` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:53.339106 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc4ed583f574cf8a12535b79d14021b)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:53.341034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:53.390463 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc4ed583f574cf8a12535b79d14021b', 'request_id': 'req-90985e07-a623-4391-914e-fccedcf1ef31'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.396990 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc4ed583f574cf8a12535b79d14021b', 'request_id': 'req-90985e07-a623-4391-914e-fccedcf1ef31'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.402996 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc4ed583f574cf8a12535b79d14021b', 'request_id': 'req-90985e07-a623-4391-914e-fccedcf1ef31'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.417879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] The token cache is being invalidate because project 5dc4ed583f574cf8a12535b79d14021b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:53.418104 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.418760 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.420333 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1398/2792] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:53 2022] DELETE /identity/v3/projects/5dc4ed583f574cf8a12535b79d14021b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:53.424583 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:53.426026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:53.439246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a254018-5b67-439d-b878-3418223fd9a6 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:53.498234 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:53.498890 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:53.499166 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:53.499433 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] PATH_INFO: `/v3/users/2a67185bb2ec4c2681ec64890cb0ed3e` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:53.500606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a67185bb2ec4c2681ec64890cb0ed3e)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:53.502516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:53.524069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:53.551414 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a67185bb2ec4c2681ec64890cb0ed3e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.552102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a67185bb2ec4c2681ec64890cb0ed3e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.558851 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a67185bb2ec4c2681ec64890cb0ed3e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.559481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a67185bb2ec4c2681ec64890cb0ed3e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.573673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-9ac12a9d-622c-4571-b60a-2d053dfe0916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a67185bb2ec4c2681ec64890cb0ed3e'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.587911 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1395/2793] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:53 2022] DELETE /identity/v3/users/2a67185bb2ec4c2681ec64890cb0ed3e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:53.767355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:53.768944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:53.782341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90985e07-a623-4391-914e-fccedcf1ef31 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:53.839856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:53.840519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:53.840795 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:53.841065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] PATH_INFO: `/v3/projects/93a119c411ed4796892b829a02ccf43d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:53.842224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a119c411ed4796892b829a02ccf43d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:53.844277 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:53.881473 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:53.895380 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a119c411ed4796892b829a02ccf43d', 'request_id': 'req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.901476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a119c411ed4796892b829a02ccf43d', 'request_id': 'req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.907080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a119c411ed4796892b829a02ccf43d', 'request_id': 'req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.921057 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] The token cache is being invalidate because project 93a119c411ed4796892b829a02ccf43d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:53.921328 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.922082 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-d24c823f-2cf3-44cd-9d36-02b8273c8a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:53.923599 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1399/2794] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:53 2022] DELETE /identity/v3/projects/93a119c411ed4796892b829a02ccf43d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:53.930129 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2ef9d-c3b6-43a7-ba61-dd62cbdc2808 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:53.930643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2ef9d-c3b6-43a7-ba61-dd62cbdc2808 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:53.930748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2ef9d-c3b6-43a7-ba61-dd62cbdc2808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:53.956728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-91c2ef9d-c3b6-43a7-ba61-dd62cbdc2808 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:53.957870 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-91c2ef9d-c3b6-43a7-ba61-dd62cbdc2808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:54.029345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1396/2795] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:54.039666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ceec5-426a-497e-998f-9ee26351ed46 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.040059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ceec5-426a-497e-998f-9ee26351ed46 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.040442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ceec5-426a-497e-998f-9ee26351ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.062681 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-e40ceec5-426a-497e-998f-9ee26351ed46 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:54.063919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e40ceec5-426a-497e-998f-9ee26351ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:54.073364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1400/2796] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:54.077837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2ef9d-c3b6-43a7-ba61-dd62cbdc2808 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:54.088093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2ef9d-c3b6-43a7-ba61-dd62cbdc2808 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:54.093172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:54.093767 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.094041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.094302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.094969 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:54.095268 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:54.096909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:54.108854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1397/2797] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:54.113300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ceec5-426a-497e-998f-9ee26351ed46 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:54.123197 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ceec5-426a-497e-998f-9ee26351ed46 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:54.128291 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:54.128885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.129160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.129430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.130058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:54.131958 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:54.161327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1401/2798] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:54 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:54.165530 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:54.175956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ab874-8a0a-4829-a7f4-48adce965bce admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:54.231199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:54.231735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.232044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.232317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.232948 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:54.234988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:54.259697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1398/2799] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:54 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:54.264121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:54.277209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2928d-e4c3-401a-98ef-d17c0db372b3 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:54.284766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:54.285378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.285649 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.285912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.286883 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:54.288517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:54.299410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1402/2800] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:54.303677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:54.314127 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7bd73-e78f-40e9-ac5f-109619bdef1a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:54.319153 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e6038-4b40-4a17-8526-33ab709581d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:54.319752 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e6038-4b40-4a17-8526-33ab709581d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.320063 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e6038-4b40-4a17-8526-33ab709581d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.320307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e6038-4b40-4a17-8526-33ab709581d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.320979 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e6038-4b40-4a17-8526-33ab709581d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:54.322559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e6038-4b40-4a17-8526-33ab709581d8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:54.333502 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1399/2801] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:54.337852 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:54.348728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b288e8a-5587-4354-824d-295f55ceccf4 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:54.355357 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:54.356016 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.356292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.356554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] PATH_INFO: `/v3/projects/683cbe3f6af646fc8c5d90e37e1e80a4/users/791abc60d36744feac98fb0660f01254/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.375673 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=683cbe3f6af646fc8c5d90e37e1e80a4, user_id=791abc60d36744feac98fb0660f01254, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:54.378351 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:54.394235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1403/2802] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:54 2022] PUT /identity/v3/projects/683cbe3f6af646fc8c5d90e37e1e80a4/users/791abc60d36744feac98fb0660f01254/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:54.404784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16234c0-3783-49e7-b594-17f4ac0e73de None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.405140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16234c0-3783-49e7-b594-17f4ac0e73de None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.405490 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16234c0-3783-49e7-b594-17f4ac0e73de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.414214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:54.436239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-a16234c0-3783-49e7-b594-17f4ac0e73de None None] MFA Rules not processed for user `791abc60d36744feac98fb0660f01254`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:54.437460 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-a16234c0-3783-49e7-b594-17f4ac0e73de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:54.505187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1400/2803] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:54.516474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:54.518260 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:54.575056 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfba620-6685-4f5b-869d-3ad15cc596ea admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:54.579510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94066b98-9023-42da-9913-a6a1c7e5200d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:54.580149 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94066b98-9023-42da-9913-a6a1c7e5200d service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:54.580430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94066b98-9023-42da-9913-a6a1c7e5200d service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:54.580699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94066b98-9023-42da-9913-a6a1c7e5200d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:54.589615 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94066b98-9023-42da-9913-a6a1c7e5200d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:54.591053 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94066b98-9023-42da-9913-a6a1c7e5200d service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:54.600394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1404/2804] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:57.216704 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16234c0-3783-49e7-b594-17f4ac0e73de None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.286038 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16234c0-3783-49e7-b594-17f4ac0e73de None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.290957 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.291597 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.291869 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.292172 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] PATH_INFO: `/v3/users/791abc60d36744feac98fb0660f01254` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.293443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=791abc60d36744feac98fb0660f01254)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.295377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.347227 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791abc60d36744feac98fb0660f01254'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.347842 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791abc60d36744feac98fb0660f01254'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.354495 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791abc60d36744feac98fb0660f01254'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.355105 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791abc60d36744feac98fb0660f01254'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.368009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791abc60d36744feac98fb0660f01254'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.383605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1401/2805] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:57 2022] DELETE /identity/v3/users/791abc60d36744feac98fb0660f01254 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:57.388348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.388814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.388913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.408100 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:57.409290 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:57.463141 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:57.474327 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1405/2806] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:57.487290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.488996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:57.504336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:57.548444 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c65b29-5806-4902-9179-352c663614a8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.552287 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf1a9c-3c74-4366-9e4c-b83ec0456b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.552893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf1a9c-3c74-4366-9e4c-b83ec0456b27 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.553168 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf1a9c-3c74-4366-9e4c-b83ec0456b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.553430 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf1a9c-3c74-4366-9e4c-b83ec0456b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.562903 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf1a9c-3c74-4366-9e4c-b83ec0456b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.564442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf1a9c-3c74-4366-9e4c-b83ec0456b27 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.573637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1402/2807] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:57.857979 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.859554 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:57.872728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64a10d-24af-44dd-88b3-b3183d1aced0 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.879005 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.879878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.880263 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.880646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] PATH_INFO: `/v3/projects/683cbe3f6af646fc8c5d90e37e1e80a4` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.882119 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683cbe3f6af646fc8c5d90e37e1e80a4)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.884170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.934359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683cbe3f6af646fc8c5d90e37e1e80a4', 'request_id': 'req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.940384 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683cbe3f6af646fc8c5d90e37e1e80a4', 'request_id': 'req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.946099 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683cbe3f6af646fc8c5d90e37e1e80a4', 'request_id': 'req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.959722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] The token cache is being invalidate because project 683cbe3f6af646fc8c5d90e37e1e80a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:57.960042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.960643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-0dc4dc0f-2b7d-46ac-8186-29962e6e31a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.962155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1406/2808] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:57 2022] DELETE /identity/v3/projects/683cbe3f6af646fc8c5d90e37e1e80a4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:57.967091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b02e2-b86c-499a-925e-c3d86d8fa77b None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.967376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b02e2-b86c-499a-925e-c3d86d8fa77b None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.967650 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b02e2-b86c-499a-925e-c3d86d8fa77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.989119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-b23b02e2-b86c-499a-925e-c3d86d8fa77b None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:57.990316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b23b02e2-b86c-499a-925e-c3d86d8fa77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.064352 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1403/2809] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:58.072233 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31ada0-9028-4aab-8532-b4f5918d7e60 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.072500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31ada0-9028-4aab-8532-b4f5918d7e60 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.072766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31ada0-9028-4aab-8532-b4f5918d7e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.092783 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-1a31ada0-9028-4aab-8532-b4f5918d7e60 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:58.094010 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1a31ada0-9028-4aab-8532-b4f5918d7e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.102519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1407/2810] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:58.106868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b02e2-b86c-499a-925e-c3d86d8fa77b None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.117797 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b02e2-b86c-499a-925e-c3d86d8fa77b None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.123092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.123689 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.123996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.124269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.124916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:58.125215 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.126863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.138480 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1404/2811] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:58.142937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31ada0-9028-4aab-8532-b4f5918d7e60 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.152458 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31ada0-9028-4aab-8532-b4f5918d7e60 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.157801 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.158399 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.158676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.158944 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.159572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.161482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.190766 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1408/2812] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:58.195118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.205571 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf3750-93ae-467d-9317-4a3e1a6c5872 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.263984 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.264629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.264846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.265107 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.265726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.267664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.291273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1405/2813] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:58 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:58.295692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.306558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d2a13-cca4-4700-b85d-c1ebc6b043c7 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.311689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.312319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.312593 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.312856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.313532 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.315178 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.325608 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1409/2814] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:58.329762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.340260 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790d2ba-16ed-4a8b-918e-bb18c57e7a13 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.346259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c9314-e54b-43b2-865a-42c92494a2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.346845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c9314-e54b-43b2-865a-42c92494a2ee admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.347116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c9314-e54b-43b2-865a-42c92494a2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.347375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c9314-e54b-43b2-865a-42c92494a2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.348044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c9314-e54b-43b2-865a-42c92494a2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.349631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c9314-e54b-43b2-865a-42c92494a2ee admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.360802 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1406/2815] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:58.365432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.377004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57dab5-e1b5-4e39-83ef-fa4808423f04 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.383366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.384089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.384366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.384623 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] PATH_INFO: `/v3/projects/1c3592efc4ea4562a7b38a9189c7d04f/users/6cfec539e6dc404fa692139310733d31/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.407778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3592efc4ea4562a7b38a9189c7d04f, user_id=6cfec539e6dc404fa692139310733d31, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.412089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.429266 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1410/2816] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:58 2022] PUT /identity/v3/projects/1c3592efc4ea4562a7b38a9189c7d04f/users/6cfec539e6dc404fa692139310733d31/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:58.435292 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e36832-405a-43c2-b249-66855e1486d1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.435576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e36832-405a-43c2-b249-66855e1486d1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.435840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e36832-405a-43c2-b249-66855e1486d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.456073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-22e36832-405a-43c2-b249-66855e1486d1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:58.457354 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-22e36832-405a-43c2-b249-66855e1486d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.475083 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:58.520512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1407/2817] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:58.530726 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.532831 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.594332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354ecdb-7530-492a-b3fc-0a96cbc2d593 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.598823 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.599425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.599688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.600011 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.609812 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.611261 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.621094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1411/2818] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:02.498483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd85e1-41b2-4a82-80c9-b8aa7ebf8aef None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:02.498992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd85e1-41b2-4a82-80c9-b8aa7ebf8aef None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:02.499095 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd85e1-41b2-4a82-80c9-b8aa7ebf8aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:02.535871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-58cd85e1-41b2-4a82-80c9-b8aa7ebf8aef None None] MFA Rules not processed for user `6cfec539e6dc404fa692139310733d31`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:02.537273 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-58cd85e1-41b2-4a82-80c9-b8aa7ebf8aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:02.604954 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1408/2819] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:04:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:02.609092 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:02.619513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d33310-5efc-4742-9fbb-43ee48777d6f service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:02.624354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:02.624984 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:02.625306 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:02.625710 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:02.626364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:02.628358 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:02.673209 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:02.683528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1412/2820] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:02 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:04:02.684805 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd85e1-41b2-4a82-80c9-b8aa7ebf8aef None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:02.695649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd85e1-41b2-4a82-80c9-b8aa7ebf8aef None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:02.758128 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:02.758891 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:02.759007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:02.759254 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:02.759906 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:02.761855 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:02.790491 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1409/2821] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:02 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:04:02.794501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:02.805371 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8455-ce89-4eab-97cf-c4953b6f425c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:02.810361 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:02.810971 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:02.811249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:02.811518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:02.812369 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:02.814074 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:02.825825 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1413/2822] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:04:02.830241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:02.841094 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1087a-e484-4c75-816c-816b313014f1 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:02.845978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:02.846651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:02.846929 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:02.847199 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] PATH_INFO: `/v3/projects/6a049abb4bc646fb971ac6ba74343785/users/6f00b73f49bc48c6a3b9dcf3151ea405/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:02.863163 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a049abb4bc646fb971ac6ba74343785, user_id=6f00b73f49bc48c6a3b9dcf3151ea405, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:02.865872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:02.881584 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1410/2823] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:02 2022] PUT /identity/v3/projects/6a049abb4bc646fb971ac6ba74343785/users/6f00b73f49bc48c6a3b9dcf3151ea405/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:04:02.885659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:02.896379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80673-4f40-425c-ad83-d68054530a17 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:02.957121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec813696-e64f-4fc6-b93a-acc1173dcfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:02.957737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813696-e64f-4fc6-b93a-acc1173dcfdd admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:02.958019 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813696-e64f-4fc6-b93a-acc1173dcfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:02.958287 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813696-e64f-4fc6-b93a-acc1173dcfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:02.958929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec813696-e64f-4fc6-b93a-acc1173dcfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:02.960558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec813696-e64f-4fc6-b93a-acc1173dcfdd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:02.977173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1414/2824] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:02.978369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:02.989073 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46674b-a596-4321-b8de-c1c67da32fd7 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:02.994161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:02.994819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:02.995092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:02.995357 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] PATH_INFO: `/v3/projects/6a049abb4bc646fb971ac6ba74343785/users/6f00b73f49bc48c6a3b9dcf3151ea405/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:02.997331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a049abb4bc646fb971ac6ba74343785, user_id=6f00b73f49bc48c6a3b9dcf3151ea405, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:03.000001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:03.011075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1411/2825] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:02 2022] PUT /identity/v3/projects/6a049abb4bc646fb971ac6ba74343785/users/6f00b73f49bc48c6a3b9dcf3151ea405/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:07.250591 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94621f34-0a0a-46cc-ad6c-5f17d9e2ef06 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:07.251179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94621f34-0a0a-46cc-ad6c-5f17d9e2ef06 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:07.251281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94621f34-0a0a-46cc-ad6c-5f17d9e2ef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:07.279278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-94621f34-0a0a-46cc-ad6c-5f17d9e2ef06 None None] MFA Rules not processed for user `6f00b73f49bc48c6a3b9dcf3151ea405`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:07.280537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-94621f34-0a0a-46cc-ad6c-5f17d9e2ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:07.350631 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1415/2826] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:04:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:07.373115 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:07.378647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:07.384807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:07.460005 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3a106-ca6a-4a9b-bf84-684c314e732a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:07.466716 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:07.467329 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:07.467606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:07.467897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:07.531144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:07.532961 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:07.542556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1412/2827] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:08.643116 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:08.643610 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:08.643728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:08.663622 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] MFA Rules not processed for user `6cfec539e6dc404fa692139310733d31`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:08.664865 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:08.674761 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1416/2828] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:04:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:17.538515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=54 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:18.658375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=71 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:31.746814 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:31.749093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:31.813585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7aea-f2e7-4a96-b956-709a24921b11 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:31.817997 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:31.818596 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:31.818873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:31.819140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:31.821261 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:31.831286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:31.832734 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:31.840856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1413/2829] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:32.311711 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:32.313430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:32.390412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bc58d-2a7b-4894-860a-062f62afa6ca None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:32.396140 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:32.396752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:32.397026 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:32.397285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:32.399245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:32.410568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:32.412163 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:32.422038 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1417/2830] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:35.105995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:35.107627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:35.125827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c865b15-5db7-400e-a796-a7d17b79e896 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:35.131699 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:35.132336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:35.132694 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:35.132960 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:35.134787 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:35.148649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:35.150266 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:35.161990 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1414/2831] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:42.417722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=32 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:45.156653 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=32 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:46.543533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.556918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886e03b-e611-4592-a3c1-60b0ab8cb6b8 service placement] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.562332 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79ac27-be1d-4663-bbb2-89b6d687295f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.562941 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ac27-be1d-4663-bbb2-89b6d687295f service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.563224 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ac27-be1d-4663-bbb2-89b6d687295f service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.563494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ac27-be1d-4663-bbb2-89b6d687295f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.565280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-dc79ac27-be1d-4663-bbb2-89b6d687295f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:46.577636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79ac27-be1d-4663-bbb2-89b6d687295f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.579198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79ac27-be1d-4663-bbb2-89b6d687295f service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.589974 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1418/2832] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:50.306411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:50.317180 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1400b-df18-4818-8742-cd1143d0669b service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:50.321880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:50.322539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:50.322819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:50.323092 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:50.333463 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:50.334934 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:50.345336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1415/2833] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:51.890219 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02164f41-4d3c-40b5-bdea-863b2a934a58 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:51.890860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02164f41-4d3c-40b5-bdea-863b2a934a58 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:51.890860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02164f41-4d3c-40b5-bdea-863b2a934a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:51.916730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-02164f41-4d3c-40b5-bdea-863b2a934a58 None None] MFA Rules not processed for user `3ec88da1704d44fb924fa3cc04e5fe25`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:51.918192 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-02164f41-4d3c-40b5-bdea-863b2a934a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:51.982558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1419/2834] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:04:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:05:00.340282 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=6 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:01.975402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=31 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:04.653564 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:04.665660 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7328-0c90-4cd5-b7e6-ecc82829294b service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:04.670893 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:04.671591 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:04.671871 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:04.672173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:04.673993 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:04.686272 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:04.687863 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:04.699843 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1416/2835] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:05:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:05:08.506480 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02164f41-4d3c-40b5-bdea-863b2a934a58 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:08.516300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02164f41-4d3c-40b5-bdea-863b2a934a58 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:08.519826 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:08.520464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:08.520744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:08.521021 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:08.522735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:08.587085 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:08.588568 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:08.598517 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1420/2836] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:05:14.693985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=6 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:14.707859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:14.709606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:14.732021 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045626c-3ae9-4fa4-81dd-bc3b508c5e13 service placement] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:14.738466 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:14.739119 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:14.739398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:14.739670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] PATH_INFO: `/v3/users/d8d329978baf425e954b9965ae0cb652` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:14.756494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d329978baf425e954b9965ae0cb652)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:14.758456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:14.810784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d329978baf425e954b9965ae0cb652'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:14.811425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d329978baf425e954b9965ae0cb652'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:14.821880 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d329978baf425e954b9965ae0cb652'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:14.822627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d329978baf425e954b9965ae0cb652'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:14.834396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d329978baf425e954b9965ae0cb652'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:14.848582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1417/2837] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:05:14 2022] DELETE /identity/v3/users/d8d329978baf425e954b9965ae0cb652 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:05:15.023108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:15.033534 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5ef10-fba5-4c9f-abaf-e354a1a57de4 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:15.092225 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:15.092842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:15.093120 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:15.093387 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] PATH_INFO: `/v3/projects/881a40342cb84608b961e6b32dceadb4` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:15.103545 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=881a40342cb84608b961e6b32dceadb4)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:15.105520 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:15.157790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881a40342cb84608b961e6b32dceadb4', 'request_id': 'req-e0b7af80-62d9-47f4-a400-812464188963'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.164675 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881a40342cb84608b961e6b32dceadb4', 'request_id': 'req-e0b7af80-62d9-47f4-a400-812464188963'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.170904 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881a40342cb84608b961e6b32dceadb4', 'request_id': 'req-e0b7af80-62d9-47f4-a400-812464188963'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.186485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] The token cache is being invalidate because project 881a40342cb84608b961e6b32dceadb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:05:15.186770 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.187392 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.188918 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1421/2838] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:05:15 2022] DELETE /identity/v3/projects/881a40342cb84608b961e6b32dceadb4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:05:15.193090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:15.194516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:15.207512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e98ca-e078-4209-9726-9365fb171a82 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:15.264017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:15.264730 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:15.265006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:15.265276 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] PATH_INFO: `/v3/users/1429c21cc23a440db90d172c38c92b21` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:15.278640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1429c21cc23a440db90d172c38c92b21)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:15.280612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:15.333415 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1429c21cc23a440db90d172c38c92b21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.334052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1429c21cc23a440db90d172c38c92b21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.338403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:15.341472 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1429c21cc23a440db90d172c38c92b21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.342284 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1429c21cc23a440db90d172c38c92b21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.355672 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1429c21cc23a440db90d172c38c92b21'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.370308 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1418/2839] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:05:15 2022] DELETE /identity/v3/users/1429c21cc23a440db90d172c38c92b21 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:05:15.562603 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:15.564161 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:15.579937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7af80-62d9-47f4-a400-812464188963 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:15.592579 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:15.641411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:15.642077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:15.642388 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:15.642665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] PATH_INFO: `/v3/projects/7799aa2d333849f6a64bd45850efb3f5` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:15.654418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7799aa2d333849f6a64bd45850efb3f5)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:15.656377 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:15.702860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7799aa2d333849f6a64bd45850efb3f5', 'request_id': 'req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.710724 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7799aa2d333849f6a64bd45850efb3f5', 'request_id': 'req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.716938 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7799aa2d333849f6a64bd45850efb3f5', 'request_id': 'req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.731578 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] The token cache is being invalidate because project 7799aa2d333849f6a64bd45850efb3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:05:15.731842 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.732575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fe74bd-6373-4e8c-9a84-0df74a29be6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:05:15.734101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1422/2840] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:05:15 2022] DELETE /identity/v3/projects/7799aa2d333849f6a64bd45850efb3f5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:05:25.367090 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=10,keystone:DELETE=3,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:25.728886 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:06:33.821963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:06:33.823512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:33.898326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34676-53df-4ba1-9507-bfca2cddb91e admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:06:33.949576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:06:33.950241 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:06:33.950474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:06:33.950747 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:06:33.952629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:34.033927 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:06:34.035391 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:06:34.047456 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1419/2841] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:06:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:06:37.106513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:06:37.107042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:06:37.107042 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:06:37.126207 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] MFA Rules not processed for user `3ec88da1704d44fb924fa3cc04e5fe25`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:06:37.127402 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:37.196949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1423/2842] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:06:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:06:37.850801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:06:37.852326 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:37.863356 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ce870-b2ed-4d01-b89b-45be3d683586 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:06:37.867512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:06:37.868140 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:06:37.868393 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:06:37.868673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:06:37.870401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:37.884291 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:06:37.885713 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:06:37.896963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1420/2843] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:06:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:06:47.187436 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=25 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:06:47.892087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=80 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:06:51.006001 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:06:51.007636 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:51.025519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d842-e739-42f4-ac6d-ab5df29f1228 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:06:51.032714 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:06:51.033344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:06:51.033618 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:06:51.033891 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:06:51.035735 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:51.048509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:06:51.050090 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:06:51.060748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1424/2844] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:07:01.056228 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=6 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:07:27.411205 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:27.420111 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d18fa2-689c-488b-8da2-9511a7b10730 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:27.423719 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:27.424349 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:27.424622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:27.424859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:27.426598 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:27.436856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:27.438263 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:27.446670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1421/2845] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:07:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:07:27.558246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:27.559803 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:27.572072 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79117c30-5d18-453d-a31f-3248d3a531f0 service nova] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:27.576262 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c49b94-1923-4356-abbd-887d8ad679e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:27.576872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49b94-1923-4356-abbd-887d8ad679e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:27.577150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49b94-1923-4356-abbd-887d8ad679e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:27.577418 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49b94-1923-4356-abbd-887d8ad679e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:27.579180 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b9c49b94-1923-4356-abbd-887d8ad679e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:27.593037 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c49b94-1923-4356-abbd-887d8ad679e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:27.594450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c49b94-1923-4356-abbd-887d8ad679e9 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:27.605202 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1425/2846] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:07:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:07:37.443242 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=6 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:07:37.600595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=6 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:08:10.138514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:08:10.152994 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba857-4c3b-4c67-b39c-ba74d6530381 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:08:10.159773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:08:10.160574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:08:10.160846 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:08:10.161401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:08:10.163323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:08:10.175302 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:08:10.177031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:08:10.187118 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1422/2847] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:08:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:08:13.281970 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117e3b6-3807-4354-ae49-c982b0420d11 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:08:13.282523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117e3b6-3807-4354-ae49-c982b0420d11 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:08:13.282523 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117e3b6-3807-4354-ae49-c982b0420d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:08:13.301719 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-c117e3b6-3807-4354-ae49-c982b0420d11 None None] MFA Rules not processed for user `3ec88da1704d44fb924fa3cc04e5fe25`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:08:13.302928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c117e3b6-3807-4354-ae49-c982b0420d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:08:13.310757 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1426/2848] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:08:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:08:13.623441 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:08:13.634143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70963cd2-d985-46fb-80b2-8a0c43570cba service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:08:13.638776 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:08:13.639375 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:08:13.639651 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:08:13.639943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:08:13.641912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:08:13.652023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:08:13.653454 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:08:13.661950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1423/2849] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:08:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:08:15.941265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117e3b6-3807-4354-ae49-c982b0420d11 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:08:15.951928 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117e3b6-3807-4354-ae49-c982b0420d11 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:08:15.956426 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:08:15.956991 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:08:15.957319 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:08:15.957499 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:08:15.959435 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:08:15.969880 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:08:15.971314 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:08:15.980089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1427/2850] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:08:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 09:08:23.658052 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=12 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:08:25.976110 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=9 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:12.905881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:12.916330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38f060-5d5b-42d9-8e04-dce1ab68cf0b service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:12.920912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:12.921513 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:12.921788 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:12.922055 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:12.924262 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:12.990043 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:12.991492 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:13.000570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1424/2851] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:09:17.799729 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:17.809017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202e87-8302-4c43-a708-673d11cbd6da service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:17.813188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:17.813790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:17.814065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:17.814331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:17.816271 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:17.880481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:17.881902 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:17.892485 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1428/2852] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:09:21.011879 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:21.013595 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:21.113605 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2b14a-8e9f-4fa8-92c7-9e4aec0da7b7 service neutron] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:21.119044 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:21.119661 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:21.119991 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:21.120265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:21.121992 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:21.134377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:21.136602 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:21.148086 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1425/2853] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 09:09:24.216250 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:24.227464 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d03de8-0d39-4f94-9bee-ef698640357f service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:24.232866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:24.233472 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:24.233745 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:24.234012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:24.235723 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:24.247425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:24.249188 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:24.260412 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1429/2854] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:09:24.275618 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:24.277193 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:24.291403 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9ea26-a89d-4ea2-970f-84437e8c038e service placement] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:24.296238 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:24.297001 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:24.297297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:24.297769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] PATH_INFO: `/v3/users/6cfec539e6dc404fa692139310733d31` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:24.299050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cfec539e6dc404fa692139310733d31)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:24.301213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:24.350467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cfec539e6dc404fa692139310733d31'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.351103 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cfec539e6dc404fa692139310733d31'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.358311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cfec539e6dc404fa692139310733d31'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.358921 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cfec539e6dc404fa692139310733d31'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.371003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cfec539e6dc404fa692139310733d31'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.382471 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:24.386840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1426/2855] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:09:24 2022] DELETE /identity/v3/users/6cfec539e6dc404fa692139310733d31 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:24.571430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:24.582198 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c23e9-e52f-4d59-9f37-b326fdfb7186 service placement] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:24.641701 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:24.642478 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:24.642775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:24.643047 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] PATH_INFO: `/v3/projects/1c3592efc4ea4562a7b38a9189c7d04f` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:24.644244 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3592efc4ea4562a7b38a9189c7d04f)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:24.646183 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:24.694549 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3592efc4ea4562a7b38a9189c7d04f', 'request_id': 'req-a1ade197-6fb7-4832-b0b2-b106f93ef21d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.700665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3592efc4ea4562a7b38a9189c7d04f', 'request_id': 'req-a1ade197-6fb7-4832-b0b2-b106f93ef21d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.706431 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3592efc4ea4562a7b38a9189c7d04f', 'request_id': 'req-a1ade197-6fb7-4832-b0b2-b106f93ef21d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.724854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] The token cache is being invalidate because project 1c3592efc4ea4562a7b38a9189c7d04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:09:24.725150 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.725787 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.727318 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1430/2856] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:09:24 2022] DELETE /identity/v3/projects/1c3592efc4ea4562a7b38a9189c7d04f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:24.731570 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:24.733122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:24.746826 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350154b9-b501-4ce8-9825-3f2c6313d0ee admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:24.804844 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:24.805485 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:24.805764 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:24.806030 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] PATH_INFO: `/v3/users/6f00b73f49bc48c6a3b9dcf3151ea405` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:24.807225 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f00b73f49bc48c6a3b9dcf3151ea405)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:24.809194 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:24.865900 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f00b73f49bc48c6a3b9dcf3151ea405'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.866637 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f00b73f49bc48c6a3b9dcf3151ea405'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.874655 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f00b73f49bc48c6a3b9dcf3151ea405'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.875381 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f00b73f49bc48c6a3b9dcf3151ea405'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.888274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f00b73f49bc48c6a3b9dcf3151ea405'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:24.903436 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1427/2857] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:09:24 2022] DELETE /identity/v3/users/6f00b73f49bc48c6a3b9dcf3151ea405 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:25.086546 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:25.088189 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:25.101535 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade197-6fb7-4832-b0b2-b106f93ef21d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:25.111152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:25.166926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:25.167569 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:25.167820 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:25.168118 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] PATH_INFO: `/v3/projects/6a049abb4bc646fb971ac6ba74343785` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:25.169365 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a049abb4bc646fb971ac6ba74343785)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:25.171278 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:25.229227 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a049abb4bc646fb971ac6ba74343785', 'request_id': 'req-2815c9d3-9be1-4b39-a182-73fd3e71b222'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:25.235425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a049abb4bc646fb971ac6ba74343785', 'request_id': 'req-2815c9d3-9be1-4b39-a182-73fd3e71b222'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:25.241700 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a049abb4bc646fb971ac6ba74343785', 'request_id': 'req-2815c9d3-9be1-4b39-a182-73fd3e71b222'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:25.255246 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] The token cache is being invalidate because project 6a049abb4bc646fb971ac6ba74343785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:09:25.255469 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:25.256131 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:25.257643 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1431/2858] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:09:25 2022] DELETE /identity/v3/projects/6a049abb4bc646fb971ac6ba74343785 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:34.900401 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:35.252595 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:44.189639 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:44.191453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:44.296390 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b696b-8012-40d9-a83e-12ab4e72d116 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:44.385579 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf149f246c1644aa85c4c162a2e07984', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:44.386429 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:44.386881 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:44.387148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:44.388988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:44.405058 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:44.406697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:44.420047 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1428/2859] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:09:49.416142 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:49.417563 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:49.479000 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815c9d3-9be1-4b39-a182-73fd3e71b222 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:49.483644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:49.484292 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:49.484573 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:49.484840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:49.486793 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:49.550519 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:49.551878 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:49.561815 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1432/2860] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:09:52.502295 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:52.503845 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:52.518050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05c9a-cb8a-4789-bf6c-ae44f7bf8f67 service nova] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:52.522638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:52.523259 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:52.523533 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:52.523801 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] PATH_INFO: `/v3/users/3ec88da1704d44fb924fa3cc04e5fe25` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:52.524926 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec88da1704d44fb924fa3cc04e5fe25)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:52.526935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:52.580916 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec88da1704d44fb924fa3cc04e5fe25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.581669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec88da1704d44fb924fa3cc04e5fe25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.589376 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec88da1704d44fb924fa3cc04e5fe25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.590075 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec88da1704d44fb924fa3cc04e5fe25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.602344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-99d0639b-a19f-4d63-ae7f-e39018806abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec88da1704d44fb924fa3cc04e5fe25'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.606167 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:52.619657 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1429/2861] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:09:52 2022] DELETE /identity/v3/users/3ec88da1704d44fb924fa3cc04e5fe25 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:52.797153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:52.808913 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e41c6-3fec-40d4-8def-f9feaad1894f service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:52.873095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:52.873737 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:52.874017 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:52.874285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] PATH_INFO: `/v3/projects/89b56ee603d34a5f85f7e007b2fe668f` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:52.875486 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b56ee603d34a5f85f7e007b2fe668f)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:52.877444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:52.926206 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b56ee603d34a5f85f7e007b2fe668f', 'request_id': 'req-c3852f0b-85e9-41fe-a659-c67e4974d93a'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.928012 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:52.932670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b56ee603d34a5f85f7e007b2fe668f', 'request_id': 'req-c3852f0b-85e9-41fe-a659-c67e4974d93a'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.938887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b56ee603d34a5f85f7e007b2fe668f', 'request_id': 'req-c3852f0b-85e9-41fe-a659-c67e4974d93a'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.953045 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] The token cache is being invalidate because project 89b56ee603d34a5f85f7e007b2fe668f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:09:52.953329 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.953930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c3852f0b-85e9-41fe-a659-c67e4974d93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:52.955420 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1433/2862] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:09:52 2022] DELETE /identity/v3/projects/89b56ee603d34a5f85f7e007b2fe668f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:52.960437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37292c-2590-4ae7-8d2b-4e55d721adeb None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:52.960722 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37292c-2590-4ae7-8d2b-4e55d721adeb None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:52.960995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37292c-2590-4ae7-8d2b-4e55d721adeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:52.981794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ee37292c-2590-4ae7-8d2b-4e55d721adeb None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:52.982988 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ee37292c-2590-4ae7-8d2b-4e55d721adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:53.041474 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1430/2863] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:09:53.049598 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a1784-7eda-46a9-9faa-9723ca04aba1 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.049854 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a1784-7eda-46a9-9faa-9723ca04aba1 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.050121 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a1784-7eda-46a9-9faa-9723ca04aba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.070162 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-082a1784-7eda-46a9-9faa-9723ca04aba1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:53.071346 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-082a1784-7eda-46a9-9faa-9723ca04aba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:53.079937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1434/2864] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:53.084156 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37292c-2590-4ae7-8d2b-4e55d721adeb None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:53.094481 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37292c-2590-4ae7-8d2b-4e55d721adeb None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:53.100547 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:53.101159 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.101435 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.101720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.102348 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:09:53.102646 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:53.104412 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:53.117106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1431/2865] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:09:53.121767 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a1784-7eda-46a9-9faa-9723ca04aba1 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:53.131574 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a1784-7eda-46a9-9faa-9723ca04aba1 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:53.137937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:53.138550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.138829 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.139096 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.139732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:53.141660 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:53.174154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1435/2866] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:09:53 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:53.178388 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:53.191245 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750d708-0fc1-4f3c-b0a0-15e413f697b3 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:53.249754 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:53.250367 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.250643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.250945 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.251537 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:53.253477 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:53.279615 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1432/2867] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:09:53 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:53.283901 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:53.294394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc5c99-18a9-4e3d-9506-46b7dd26acb7 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:53.299359 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:53.299989 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.300265 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.300533 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.301175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:53.302747 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:53.312851 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1436/2868] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:53.316775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:53.326940 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44599-5314-4db1-bb7c-f88037dbf80a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:53.331854 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6f69b-e025-4dd3-b804-b7e73586fc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:53.332465 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f69b-e025-4dd3-b804-b7e73586fc89 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.332738 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f69b-e025-4dd3-b804-b7e73586fc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.333004 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f69b-e025-4dd3-b804-b7e73586fc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.333670 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6f69b-e025-4dd3-b804-b7e73586fc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:53.335244 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6f69b-e025-4dd3-b804-b7e73586fc89 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:53.349868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1433/2869] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:53.356228 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:53.369717 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080e39-98cf-45f4-ba73-f99cf9f97b26 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:53.374808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:53.375456 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.375733 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.376035 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] PATH_INFO: `/v3/projects/91465aa3892e4ab2b4b08930703da082/users/7763199015a04bf38708b7475a5e8f40/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.391270 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91465aa3892e4ab2b4b08930703da082, user_id=7763199015a04bf38708b7475a5e8f40, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:53.393942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:53.409013 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1437/2870] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:09:53 2022] PUT /identity/v3/projects/91465aa3892e4ab2b4b08930703da082/users/7763199015a04bf38708b7475a5e8f40/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:09:53.414123 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687faad8-508e-4702-acf5-c8328635ab90 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.414442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687faad8-508e-4702-acf5-c8328635ab90 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.414697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687faad8-508e-4702-acf5-c8328635ab90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.435512 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-687faad8-508e-4702-acf5-c8328635ab90 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:53.436832 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-687faad8-508e-4702-acf5-c8328635ab90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:53.506943 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1434/2871] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:53.517137 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:53.519730 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:53.584297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835d2e6-a13a-49ff-b9b1-9edb9ba98b3e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:53.589474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:53.590080 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:53.590354 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:53.590619 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:53.600234 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:53.601657 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:53.610933 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1438/2872] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:09:54.535345 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d236d1-ba4f-43a7-96d4-1fbe577a51e9 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:54.536006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d236d1-ba4f-43a7-96d4-1fbe577a51e9 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:54.536006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d236d1-ba4f-43a7-96d4-1fbe577a51e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:54.543830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:54.567247 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-c1d236d1-ba4f-43a7-96d4-1fbe577a51e9 None None] MFA Rules not processed for user `7763199015a04bf38708b7475a5e8f40`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:54.568507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c1d236d1-ba4f-43a7-96d4-1fbe577a51e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:54.635575 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1435/2873] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:54 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:54.640406 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:54.650254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addecf2-11a1-4da3-a994-cc69a7093cc5 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:54.655504 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:54.656134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:54.656411 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:54.656676 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:54.657310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:54.659221 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:54.687381 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1439/2874] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:09:54 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:54.691416 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d236d1-ba4f-43a7-96d4-1fbe577a51e9 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:54.702669 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d236d1-ba4f-43a7-96d4-1fbe577a51e9 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:54.760331 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:54.760944 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:54.761226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:54.761497 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:54.762137 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:54.764071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:54.785274 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1436/2875] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:09:54 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:09:54.789475 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:54.800575 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10c7ab-7972-4631-8e96-40fa6e6b3c6f admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:54.806898 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ca413-07a6-463b-beec-43179543231d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:54.807524 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca413-07a6-463b-beec-43179543231d admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:54.807798 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca413-07a6-463b-beec-43179543231d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:54.808097 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ca413-07a6-463b-beec-43179543231d admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:54.808734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ca413-07a6-463b-beec-43179543231d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:54.810296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ca413-07a6-463b-beec-43179543231d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:54.821048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1440/2876] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:54.825248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:54.835406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec44e8b-9add-4e28-a0e3-13c2dcd9a65f admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:54.840538 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:54.841181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:54.841443 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:54.841714 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] PATH_INFO: `/v3/projects/73464987de4549288db8cf4fa4f281d0/users/cd2156b83845419eb735ee5e836b9108/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:54.857377 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73464987de4549288db8cf4fa4f281d0, user_id=cd2156b83845419eb735ee5e836b9108, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:54.860087 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:54.876909 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1437/2877] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:09:54 2022] PUT /identity/v3/projects/73464987de4549288db8cf4fa4f281d0/users/cd2156b83845419eb735ee5e836b9108/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:09:54.881015 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ca413-07a6-463b-beec-43179543231d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:54.891077 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ca413-07a6-463b-beec-43179543231d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:54.934422 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:54.949969 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb224af-edbf-4c19-b4b5-bd1e4ffe7d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:54.950665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb224af-edbf-4c19-b4b5-bd1e4ffe7d64 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:54.951002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb224af-edbf-4c19-b4b5-bd1e4ffe7d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:54.951337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb224af-edbf-4c19-b4b5-bd1e4ffe7d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:54.952033 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb224af-edbf-4c19-b4b5-bd1e4ffe7d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:54.953732 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb224af-edbf-4c19-b4b5-bd1e4ffe7d64 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:54.964044 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1441/2878] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:54.968515 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:54.979638 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e819c2-1f7b-4f28-a184-65f04a407bc5 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:54.985009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:54.985666 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:54.985942 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:54.986213 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] PATH_INFO: `/v3/projects/73464987de4549288db8cf4fa4f281d0/users/cd2156b83845419eb735ee5e836b9108/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:54.988243 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] RBAC: Authorizing `identity:create_grant(project_id=73464987de4549288db8cf4fa4f281d0, user_id=cd2156b83845419eb735ee5e836b9108, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:54.991219 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:55.002355 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1438/2879] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:09:54 2022] PUT /identity/v3/projects/73464987de4549288db8cf4fa4f281d0/users/cd2156b83845419eb735ee5e836b9108/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:55.893872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0569c-a82a-4573-93e9-263d557af6df None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:55.894421 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0569c-a82a-4573-93e9-263d557af6df None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:55.894629 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0569c-a82a-4573-93e9-263d557af6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:55.924799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f2e0569c-a82a-4573-93e9-263d557af6df None None] MFA Rules not processed for user `cd2156b83845419eb735ee5e836b9108`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:55.926029 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f2e0569c-a82a-4573-93e9-263d557af6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:56.006022 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1442/2880] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:55 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:57.027440 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:57.075562 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:57.093581 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdea5fb-d7d4-4b18-80aa-f7730cb8022a admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:57.098677 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:57.099310 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:57.099587 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:57.099856 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] PATH_INFO: `/v3/users/7763199015a04bf38708b7475a5e8f40` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:57.101026 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7763199015a04bf38708b7475a5e8f40)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:57.102956 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:57.158411 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7763199015a04bf38708b7475a5e8f40'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.159116 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7763199015a04bf38708b7475a5e8f40'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.166421 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7763199015a04bf38708b7475a5e8f40'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.167017 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7763199015a04bf38708b7475a5e8f40'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.179795 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7763199015a04bf38708b7475a5e8f40'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.194464 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1439/2881] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:09:57 2022] DELETE /identity/v3/users/7763199015a04bf38708b7475a5e8f40 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:57.374940 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0569c-a82a-4573-93e9-263d557af6df None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:57.389505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0569c-a82a-4573-93e9-263d557af6df None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:57.446588 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:57.447145 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:57.447496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:57.447762 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] PATH_INFO: `/v3/projects/91465aa3892e4ab2b4b08930703da082` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:57.448921 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] RBAC: Authorizing `identity:delete_project(project_id=91465aa3892e4ab2b4b08930703da082)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:57.450811 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:57.498748 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91465aa3892e4ab2b4b08930703da082', 'request_id': 'req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.504695 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91465aa3892e4ab2b4b08930703da082', 'request_id': 'req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.510187 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91465aa3892e4ab2b4b08930703da082', 'request_id': 'req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.520537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:57.525370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] The token cache is being invalidate because project 91465aa3892e4ab2b4b08930703da082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:09:57.526102 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.526817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.528398 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1443/2882] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:09:57 2022] DELETE /identity/v3/projects/91465aa3892e4ab2b4b08930703da082 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:57.532853 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:57.534286 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:57.547654 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6ecd0-222a-4e41-b34c-cee4f61fc5a9 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:57.603647 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:57.604314 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:57.604592 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:57.604859 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] PATH_INFO: `/v3/users/cd2156b83845419eb735ee5e836b9108` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:57.606031 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2156b83845419eb735ee5e836b9108)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:57.608025 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:57.659860 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2156b83845419eb735ee5e836b9108'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.660545 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2156b83845419eb735ee5e836b9108'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.667640 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2156b83845419eb735ee5e836b9108'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.668315 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2156b83845419eb735ee5e836b9108'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.675535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:57.681643 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-d899aa18-89d1-47fb-94d0-34ee1324a083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2156b83845419eb735ee5e836b9108'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:57.695453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1440/2883] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:09:57 2022] DELETE /identity/v3/users/cd2156b83845419eb735ee5e836b9108 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:09:57.878620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:57.880337 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:57.897809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb3430-aa04-4cf5-8f90-f6e8c0a7c61c admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:57.978875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:57.979518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:57.979792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:57.980108 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] PATH_INFO: `/v3/projects/73464987de4549288db8cf4fa4f281d0` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:57.981341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=73464987de4549288db8cf4fa4f281d0)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:57.983340 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.037324 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73464987de4549288db8cf4fa4f281d0', 'request_id': 'req-f0e97fe0-0a78-422a-8622-43ce84040d1f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:58.044344 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73464987de4549288db8cf4fa4f281d0', 'request_id': 'req-f0e97fe0-0a78-422a-8622-43ce84040d1f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:58.052515 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73464987de4549288db8cf4fa4f281d0', 'request_id': 'req-f0e97fe0-0a78-422a-8622-43ce84040d1f'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:58.067583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] The token cache is being invalidate because project 73464987de4549288db8cf4fa4f281d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:09:58.067875 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:58.068525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f0e97fe0-0a78-422a-8622-43ce84040d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:09:58.070058 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1444/2884] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:09:57 2022] DELETE /identity/v3/projects/73464987de4549288db8cf4fa4f281d0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:09:58.075239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd18e5-c2da-4157-9908-6a329eef6479 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.075765 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd18e5-c2da-4157-9908-6a329eef6479 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.075868 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd18e5-c2da-4157-9908-6a329eef6479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.095678 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-81bd18e5-c2da-4157-9908-6a329eef6479 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:58.097784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-81bd18e5-c2da-4157-9908-6a329eef6479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:58.158478 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1441/2885] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:58.166379 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852693d-39a5-4e47-b4df-52c92362e93d None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.166666 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852693d-39a5-4e47-b4df-52c92362e93d None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.166942 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852693d-39a5-4e47-b4df-52c92362e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.187276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5852693d-39a5-4e47-b4df-52c92362e93d None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:58.188543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5852693d-39a5-4e47-b4df-52c92362e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:58.196632 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1445/2886] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:58.201246 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd18e5-c2da-4157-9908-6a329eef6479 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.211050 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd18e5-c2da-4157-9908-6a329eef6479 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.216432 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.217037 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.217313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.217585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] PATH_INFO: `/v3/domains` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.218237 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:09:58.218539 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.220161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.231930 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1442/2887] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:09:58.236296 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852693d-39a5-4e47-b4df-52c92362e93d None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.245885 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852693d-39a5-4e47-b4df-52c92362e93d None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.250662 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.251258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.251537 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.251808 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.252500 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.254378 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.286876 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1446/2888] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:09:58 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:58.291236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.301507 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35ea2a-5932-4d81-82f7-164c67da2257 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.358626 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.359224 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.359503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.359769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.360424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.362335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.393099 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1443/2889] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:09:58 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:58.397482 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.408004 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6486f2-9568-449b-99cf-0d4ac5b64aac admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.412764 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.413364 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.413644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.413912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.414553 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.416276 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.426894 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1447/2890] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:58.430824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.441071 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e65d22-f968-4fe4-ae1d-9bba6eca34a8 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.445912 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb02a46-5904-4483-8605-77edb20527a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.446510 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb02a46-5904-4483-8605-77edb20527a8 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.446793 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb02a46-5904-4483-8605-77edb20527a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.447064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb02a46-5904-4483-8605-77edb20527a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.447762 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb02a46-5904-4483-8605-77edb20527a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.449398 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb02a46-5904-4483-8605-77edb20527a8 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.460089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1444/2891] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:58.464375 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.474502 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203cbab-f8c8-496e-8332-17b2e4c55b94 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.479256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.479919 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.480200 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.480470 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] PATH_INFO: `/v3/projects/24d574ac8de74835a7853db726eafa62/users/8187644a9f3f4dd2be67d28b1f097c4e/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.495790 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d574ac8de74835a7853db726eafa62, user_id=8187644a9f3f4dd2be67d28b1f097c4e, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.498432 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.514274 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1448/2892] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:09:58 2022] PUT /identity/v3/projects/24d574ac8de74835a7853db726eafa62/users/8187644a9f3f4dd2be67d28b1f097c4e/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:58.579550 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d5768-c2f5-40be-ad0a-28c707e3691a None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.579799 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d5768-c2f5-40be-ad0a-28c707e3691a None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.580122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d5768-c2f5-40be-ad0a-28c707e3691a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.611297 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-040d5768-c2f5-40be-ad0a-28c707e3691a None None] MFA Rules not processed for user `8187644a9f3f4dd2be67d28b1f097c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:58.612559 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-040d5768-c2f5-40be-ad0a-28c707e3691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:58.652821 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:58.687006 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1445/2893] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:58.691834 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.719628 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:58.757688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd22e49-1a3e-4b06-a070-0aca42ce64fa admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.762582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.763175 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.763451 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.763715 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.764348 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.766605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.810847 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1449/2894] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:09:58 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:58.815296 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d5768-c2f5-40be-ad0a-28c707e3691a None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.826339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d5768-c2f5-40be-ad0a-28c707e3691a None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.885091 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.885702 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.885976 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.886248 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.886872 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.888827 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.912798 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1446/2895] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:09:58 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:58.916934 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.927281 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde63fba-4b4e-4b33-ba61-a87dcefc5026 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.932450 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.933098 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.933370 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.933639 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.934280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.935856 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.946509 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1450/2896] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:09:58.950692 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.961897 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a74e1-e371-4652-aa11-036ea285c7fd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:58.967372 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a1a7f-66ed-4b93-a3d8-9d4be806bf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:58.968007 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a1a7f-66ed-4b93-a3d8-9d4be806bf4c admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:58.968289 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a1a7f-66ed-4b93-a3d8-9d4be806bf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:58.968556 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a1a7f-66ed-4b93-a3d8-9d4be806bf4c admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:58.969239 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a1a7f-66ed-4b93-a3d8-9d4be806bf4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:58.970824 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a1a7f-66ed-4b93-a3d8-9d4be806bf4c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:58.981768 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1447/2897] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:58.986352 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:58.997304 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b05128-2e85-42ad-ba0c-69de3eb27718 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.002557 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.003240 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] REQUEST_METHOD: `PUT` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.003518 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.003806 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] PATH_INFO: `/v3/projects/8b915a9af02f466481a82f0f323fba8c/users/ce75c3a306074e7fa46c367b2b1a8b56/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.019249 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b915a9af02f466481a82f0f323fba8c, user_id=ce75c3a306074e7fa46c367b2b1a8b56, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.021866 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.036413 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1451/2898] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:09:58 2022] PUT /identity/v3/projects/8b915a9af02f466481a82f0f323fba8c/users/ce75c3a306074e7fa46c367b2b1a8b56/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:59.046632 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a0a6-2b5f-45a2-a9e3-9da828a33757 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.046915 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a0a6-2b5f-45a2-a9e3-9da828a33757 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.047187 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a0a6-2b5f-45a2-a9e3-9da828a33757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.075089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-2408a0a6-2b5f-45a2-a9e3-9da828a33757 None None] MFA Rules not processed for user `ce75c3a306074e7fa46c367b2b1a8b56`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:59.076344 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-2408a0a6-2b5f-45a2-a9e3-9da828a33757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:59.146514 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1448/2899] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:09:59.151173 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:59.216336 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c515b0f-eb1e-470c-aa9d-a115a0e9874e admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.221307 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.221907 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.222179 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.222446 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] PATH_INFO: `/v3/projects` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.223084 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.225002 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.255570 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1452/2900] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:09:59 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:09:59.259770 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408a0a6-2b5f-45a2-a9e3-9da828a33757 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:59.270483 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408a0a6-2b5f-45a2-a9e3-9da828a33757 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.315335 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:59.330369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.331009 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.331285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.331552 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] PATH_INFO: `/v3/users` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.332214 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.334126 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.356673 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1449/2901] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:09:59 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:09:59.361908 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:59.372775 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b77126-d163-4588-b010-acbf0789965a admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.377702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.378299 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.378572 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.378840 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.379471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.381089 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.391862 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1453/2902] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:09:59.396160 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:59.407389 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18547a-71ab-4d9f-bf04-b8346936dcd5 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.412735 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.413387 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.413664 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.413933 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] PATH_INFO: `/v3/projects/de7ed4e9320049f2a17bbbf4c4d4027c/users/76724a3efb444329a1dafabf02450ad5/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.428665 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7ed4e9320049f2a17bbbf4c4d4027c, user_id=76724a3efb444329a1dafabf02450ad5, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.431330 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.460563 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1450/2903] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:09:59 2022] PUT /identity/v3/projects/de7ed4e9320049f2a17bbbf4c4d4027c/users/76724a3efb444329a1dafabf02450ad5/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:59.465528 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:59.480822 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7349d-8a6a-4905-b12d-a1728dd83d57 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.531926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:09:59.546670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f67c9-a6a6-43f1-91bb-4a339ba226dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.547279 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f67c9-a6a6-43f1-91bb-4a339ba226dd admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.547558 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f67c9-a6a6-43f1-91bb-4a339ba226dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.547828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f67c9-a6a6-43f1-91bb-4a339ba226dd admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.548501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f67c9-a6a6-43f1-91bb-4a339ba226dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.550094 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f67c9-a6a6-43f1-91bb-4a339ba226dd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.560295 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1454/2904] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 09:09:59.564750 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:59.575229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dc37d-112c-4761-9029-90fc5480e4ec admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.580459 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.581106 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.581458 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.581726 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] PATH_INFO: `/v3/projects/de7ed4e9320049f2a17bbbf4c4d4027c/users/76724a3efb444329a1dafabf02450ad5/roles/3341c8a56a534032a6359202e4ab0ba7` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.583736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7ed4e9320049f2a17bbbf4c4d4027c, user_id=76724a3efb444329a1dafabf02450ad5, role_id=3341c8a56a534032a6359202e4ab0ba7)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.586425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.596784 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1451/2905] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:09:59 2022] PUT /identity/v3/projects/de7ed4e9320049f2a17bbbf4c4d4027c/users/76724a3efb444329a1dafabf02450ad5/roles/3341c8a56a534032a6359202e4ab0ba7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:09:59.608394 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.608645 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.608939 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.637929 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] MFA Rules not processed for user `76724a3efb444329a1dafabf02450ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:09:59.639122 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:59.718607 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1455/2906] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:09:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:09:59.729749 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:09:59.731829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:09:59.792173 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94637f1a-39fb-469d-82e9-e7cfd992dc8c admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:09:59.795790 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:09:59.796428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:09:59.796706 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:09:59.796978 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:09:59.806218 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:09:59.807697 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:09:59.817104 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1452/2907] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:09:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:00.352417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:00.353922 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:00.367196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29317bc0-a723-4daf-b195-16bd955c2769 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:00.371844 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:00.372487 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:00.372772 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:00.373048 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:00.435474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:00.436961 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:00.446238 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1456/2908] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:04.545766 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:04.554898 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b0376-bc8e-48b0-a761-65e496c7f26d service cinder] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:04.558775 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1d936-5a74-4290-97fc-620d36481eb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:04.559369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d936-5a74-4290-97fc-620d36481eb1 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:04.559645 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d936-5a74-4290-97fc-620d36481eb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:04.559917 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1d936-5a74-4290-97fc-620d36481eb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:04.618952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1d936-5a74-4290-97fc-620d36481eb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:04.620396 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1d936-5a74-4290-97fc-620d36481eb1 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:04.629269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1453/2909] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:06.809476 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:06.881462 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=100 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:06.900331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f79fb-9879-4c45-9f58-46e406229331 service cinder] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:06.905297 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:06.905926 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:06.906204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:06.906474 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] PATH_INFO: `/v3/users/8187644a9f3f4dd2be67d28b1f097c4e` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:06.907627 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8187644a9f3f4dd2be67d28b1f097c4e)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:06.909605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:06.960155 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8187644a9f3f4dd2be67d28b1f097c4e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:06.960799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8187644a9f3f4dd2be67d28b1f097c4e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:06.961417 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8187644a9f3f4dd2be67d28b1f097c4e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:06.968688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8187644a9f3f4dd2be67d28b1f097c4e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:06.980734 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8187644a9f3f4dd2be67d28b1f097c4e'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:06.997635 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1457/2910] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:10:06 2022] DELETE /identity/v3/users/8187644a9f3f4dd2be67d28b1f097c4e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:07.002112 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:07.002606 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:07.002725 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:07.032733 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:07.033935 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:07.060667 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:07.096442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1454/2911] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:07.106682 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:07.108331 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:07.188310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d69285-ceeb-4524-80a2-af691e33da68 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:07.192258 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:07.192870 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:07.193146 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:07.193425 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:07.203009 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:07.204496 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:07.214062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1458/2912] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:07.490144 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:07.491631 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:07.504064 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96078de7-ae0a-41e8-9173-a0314b1189e1 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:07.509328 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:07.509952 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:07.510229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:07.510493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] PATH_INFO: `/v3/projects/24d574ac8de74835a7853db726eafa62` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:07.511789 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d574ac8de74835a7853db726eafa62)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:07.513760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:07.563925 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d574ac8de74835a7853db726eafa62', 'request_id': 'req-b9b3396b-4842-4ad9-9346-648506c15775'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.569996 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d574ac8de74835a7853db726eafa62', 'request_id': 'req-b9b3396b-4842-4ad9-9346-648506c15775'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.575746 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d574ac8de74835a7853db726eafa62', 'request_id': 'req-b9b3396b-4842-4ad9-9346-648506c15775'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.591049 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] The token cache is being invalidate because project 24d574ac8de74835a7853db726eafa62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:10:07.591332 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.591975 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.593470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1455/2913] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:10:07 2022] DELETE /identity/v3/projects/24d574ac8de74835a7853db726eafa62 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:07.597556 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:07.607471 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd3f8-2081-4843-a632-c1ddab7d42bf service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:07.656949 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:07.668683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:07.669531 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:07.670007 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:07.670539 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] PATH_INFO: `/v3/users/ce75c3a306074e7fa46c367b2b1a8b56` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:07.671728 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce75c3a306074e7fa46c367b2b1a8b56)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:07.673688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:07.723792 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce75c3a306074e7fa46c367b2b1a8b56'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.724543 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce75c3a306074e7fa46c367b2b1a8b56'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.725235 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce75c3a306074e7fa46c367b2b1a8b56'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.732688 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce75c3a306074e7fa46c367b2b1a8b56'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.745805 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce75c3a306074e7fa46c367b2b1a8b56'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:07.761251 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1459/2914] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:10:07 2022] DELETE /identity/v3/users/ce75c3a306074e7fa46c367b2b1a8b56 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:08.046210 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:08.047828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:08.061588 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3396b-4842-4ad9-9346-648506c15775 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:08.121323 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:08.121888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:08.122161 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:08.122425 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] PATH_INFO: `/v3/projects/8b915a9af02f466481a82f0f323fba8c` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:08.123548 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b915a9af02f466481a82f0f323fba8c)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:08.125500 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:08.172096 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b915a9af02f466481a82f0f323fba8c', 'request_id': 'req-1b49d4e1-9920-462d-9be3-0e1677a75ddd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.178307 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b915a9af02f466481a82f0f323fba8c', 'request_id': 'req-1b49d4e1-9920-462d-9be3-0e1677a75ddd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.180269 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:08.184399 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b915a9af02f466481a82f0f323fba8c', 'request_id': 'req-1b49d4e1-9920-462d-9be3-0e1677a75ddd'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.198950 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] The token cache is being invalidate because project 8b915a9af02f466481a82f0f323fba8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:10:08.199236 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.199847 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.201400 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1456/2915] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:10:08 2022] DELETE /identity/v3/projects/8b915a9af02f466481a82f0f323fba8c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:08.205525 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:08.207065 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:08.220442 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2776027-4f0e-4e7e-9fa7-fe2548d4aff7 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:08.280809 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:08.281430 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:08.281704 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:08.281966 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] PATH_INFO: `/v3/users/76724a3efb444329a1dafabf02450ad5` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:08.283215 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=76724a3efb444329a1dafabf02450ad5)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:08.285170 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:08.334280 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76724a3efb444329a1dafabf02450ad5'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.334906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76724a3efb444329a1dafabf02450ad5'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.335513 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76724a3efb444329a1dafabf02450ad5'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.342366 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76724a3efb444329a1dafabf02450ad5'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.353034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:08.354285 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-6d715e6a-aef2-4d21-b6b7-76f90e5ed2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76724a3efb444329a1dafabf02450ad5'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.370871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1460/2916] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:10:08 2022] DELETE /identity/v3/users/76724a3efb444329a1dafabf02450ad5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:08.642883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:08.644442 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:08.657830 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49d4e1-9920-462d-9be3-0e1677a75ddd admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:08.719516 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:08.720179 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:08.720453 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:08.720720 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] PATH_INFO: `/v3/projects/de7ed4e9320049f2a17bbbf4c4d4027c` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:08.721840 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7ed4e9320049f2a17bbbf4c4d4027c)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:08.723773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:08.772501 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7ed4e9320049f2a17bbbf4c4d4027c', 'request_id': 'req-b2a454ad-741b-401e-ba70-c90c0f73d316'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.778644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7ed4e9320049f2a17bbbf4c4d4027c', 'request_id': 'req-b2a454ad-741b-401e-ba70-c90c0f73d316'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.784202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7ed4e9320049f2a17bbbf4c4d4027c', 'request_id': 'req-b2a454ad-741b-401e-ba70-c90c0f73d316'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.799829 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] The token cache is being invalidate because project de7ed4e9320049f2a17bbbf4c4d4027c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:10:08.800141 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.800760 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-b2a454ad-741b-401e-ba70-c90c0f73d316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:08.802232 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1457/2917] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:10:08 2022] DELETE /identity/v3/projects/de7ed4e9320049f2a17bbbf4c4d4027c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:08.809410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea9de6-7a5e-451f-9637-a6e1bf889a5a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:08.809692 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea9de6-7a5e-451f-9637-a6e1bf889a5a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:08.809963 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea9de6-7a5e-451f-9637-a6e1bf889a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:08.829906 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-5dea9de6-7a5e-451f-9637-a6e1bf889a5a None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:08.831101 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-5dea9de6-7a5e-451f-9637-a6e1bf889a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:08.892415 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1461/2918] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:08.901181 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc1cb5-739a-4e28-856a-5e6e2c6a4f89 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:08.901467 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc1cb5-739a-4e28-856a-5e6e2c6a4f89 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:08.901769 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc1cb5-739a-4e28-856a-5e6e2c6a4f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:08.921759 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-92fc1cb5-739a-4e28-856a-5e6e2c6a4f89 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:08.922968 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-92fc1cb5-739a-4e28-856a-5e6e2c6a4f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:08.931428 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1458/2919] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:08.935998 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea9de6-7a5e-451f-9637-a6e1bf889a5a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:08.946119 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea9de6-7a5e-451f-9637-a6e1bf889a5a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:08.951670 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:08.952293 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:08.952600 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:08.952860 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:08.953479 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:10:08.953778 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:08.955407 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:08.975204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1462/2920] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:10:08.979695 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc1cb5-739a-4e28-856a-5e6e2c6a4f89 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:08.989883 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc1cb5-739a-4e28-856a-5e6e2c6a4f89 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:08.995148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:08.995748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:08.996078 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:08.996351 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:08.996982 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:08.999067 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:09.033143 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1459/2921] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:10:08 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:10:09.037980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:09.049302 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd03619-b5c4-47d8-8fb7-1e7a66c295bd admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:09.111705 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:09.112330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:09.112605 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:09.112871 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:09.113501 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:09.115739 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:09.138590 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1463/2922] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:10:09 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:10:09.143024 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:09.153424 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309e5b-d01d-402d-a593-75cf509f18ca admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:09.158339 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:09.158938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:09.159226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:09.159494 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:09.160176 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:09.161807 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:09.171627 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1460/2923] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:10:09.175428 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:09.185665 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db080ec7-c771-421f-a604-ac5fa5c86c81 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:09.190322 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d37b4-333a-4a9f-b82d-cfa90720c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:09.190912 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d37b4-333a-4a9f-b82d-cfa90720c963 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:09.191185 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d37b4-333a-4a9f-b82d-cfa90720c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:09.191448 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d37b4-333a-4a9f-b82d-cfa90720c963 admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:09.192154 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d37b4-333a-4a9f-b82d-cfa90720c963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:09.193754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d37b4-333a-4a9f-b82d-cfa90720c963 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:09.203872 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1464/2924] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:10:09.208022 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:09.218202 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81fe7-f0e9-41a1-86d3-5c429e900a0b admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:09.223186 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:09.223836 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:09.224216 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:09.224493 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] PATH_INFO: `/v3/projects/0813fe35a4114ea1b5bd54bd59f2ad34/users/c21a64f720e64805b2f7cdb06247f265/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:09.239861 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0813fe35a4114ea1b5bd54bd59f2ad34, user_id=c21a64f720e64805b2f7cdb06247f265, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:09.242532 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:09.257382 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1461/2925] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:10:09 2022] PUT /identity/v3/projects/0813fe35a4114ea1b5bd54bd59f2ad34/users/c21a64f720e64805b2f7cdb06247f265/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:10:09.268754 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eba1c-eef7-4b2b-b21f-500e7f146d45 None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:09.269256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eba1c-eef7-4b2b-b21f-500e7f146d45 None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:09.269256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eba1c-eef7-4b2b-b21f-500e7f146d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:09.298817 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-f87eba1c-eef7-4b2b-b21f-500e7f146d45 None None] MFA Rules not processed for user `c21a64f720e64805b2f7cdb06247f265`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:09.300087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f87eba1c-eef7-4b2b-b21f-500e7f146d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:09.353410 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:09.384134 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1465/2926] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:09.397229 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:09.399196 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:09.414560 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:09.463812 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36538e-c530-4288-a5b5-17c584bc8b29 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:09.468201 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b689b50-9c85-4379-8c43-1c5ba151c429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:09.468828 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b689b50-9c85-4379-8c43-1c5ba151c429 service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:09.469102 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b689b50-9c85-4379-8c43-1c5ba151c429 service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:09.469371 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b689b50-9c85-4379-8c43-1c5ba151c429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:09.477888 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b689b50-9c85-4379-8c43-1c5ba151c429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:09.479313 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b689b50-9c85-4379-8c43-1c5ba151c429 service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:09.488685 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1462/2927] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:19.380086 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=7 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:19.484470 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=21 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:31.125105 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eba1c-eef7-4b2b-b21f-500e7f146d45 None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:31.209510 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eba1c-eef7-4b2b-b21f-500e7f146d45 None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:31.214542 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:31.215167 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:31.215455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:31.215722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] PATH_INFO: `/v3/users/c21a64f720e64805b2f7cdb06247f265` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:31.216910 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21a64f720e64805b2f7cdb06247f265)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:31.218814 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:31.269393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21a64f720e64805b2f7cdb06247f265'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.270003 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21a64f720e64805b2f7cdb06247f265'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.270651 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21a64f720e64805b2f7cdb06247f265'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.277743 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21a64f720e64805b2f7cdb06247f265'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.290201 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21a64f720e64805b2f7cdb06247f265'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.305034 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1466/2928] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:10:31 2022] DELETE /identity/v3/users/c21a64f720e64805b2f7cdb06247f265 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:31.309794 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:31.310316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:31.310316 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:31.342228 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:31.343437 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:31.404652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1463/2929] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:31.414879 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:31.416444 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:31.490095 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de68d-4a29-4eb7-b60e-671ccb372482 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:31.493937 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b89c9-05f3-4ab5-b348-161c7971d2da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:31.494550 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b89c9-05f3-4ab5-b348-161c7971d2da service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:31.494819 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b89c9-05f3-4ab5-b348-161c7971d2da service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:31.495087 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b89c9-05f3-4ab5-b348-161c7971d2da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:31.502176 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:31.504646 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b89c9-05f3-4ab5-b348-161c7971d2da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:31.506139 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b89c9-05f3-4ab5-b348-161c7971d2da service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:31.517256 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1467/2930] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:31.788093 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:31.790069 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:31.806369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084167d8-c61c-48df-9656-62a19ec80c9c None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:31.812652 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:31.813285 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:31.813582 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:31.813938 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] PATH_INFO: `/v3/projects/0813fe35a4114ea1b5bd54bd59f2ad34` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:31.815048 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0813fe35a4114ea1b5bd54bd59f2ad34)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:31.817815 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:31.874184 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0813fe35a4114ea1b5bd54bd59f2ad34', 'request_id': 'req-03f5fa27-09be-49e4-9560-a02a6b05a3ea'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.881298 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0813fe35a4114ea1b5bd54bd59f2ad34', 'request_id': 'req-03f5fa27-09be-49e4-9560-a02a6b05a3ea'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.887662 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0813fe35a4114ea1b5bd54bd59f2ad34', 'request_id': 'req-03f5fa27-09be-49e4-9560-a02a6b05a3ea'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.902265 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] The token cache is being invalidate because project 0813fe35a4114ea1b5bd54bd59f2ad34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:10:31.902527 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.903226 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-03f5fa27-09be-49e4-9560-a02a6b05a3ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:31.904823 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1464/2931] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:10:31 2022] DELETE /identity/v3/projects/0813fe35a4114ea1b5bd54bd59f2ad34 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:31.909980 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a1d81-9926-4d39-966e-b39e6a1d966f None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:31.910268 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a1d81-9926-4d39-966e-b39e6a1d966f None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:31.910541 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a1d81-9926-4d39-966e-b39e6a1d966f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:31.930702 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-723a1d81-9926-4d39-966e-b39e6a1d966f None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:31.931951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-723a1d81-9926-4d39-966e-b39e6a1d966f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:31.990699 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1468/2932] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:31.998622 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2422ed-7d9f-4b39-be2b-ebadce1c7cf4 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:31.998907 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2422ed-7d9f-4b39-be2b-ebadce1c7cf4 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:31.999177 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2422ed-7d9f-4b39-be2b-ebadce1c7cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.019748 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-5e2422ed-7d9f-4b39-be2b-ebadce1c7cf4 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:32.020965 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-5e2422ed-7d9f-4b39-be2b-ebadce1c7cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:32.029612 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1465/2933] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:32.033836 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a1d81-9926-4d39-966e-b39e6a1d966f None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:32.043722 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a1d81-9926-4d39-966e-b39e6a1d966f None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:32.048797 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:32.049393 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.049672 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.049943 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.050582 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:10:32.050887 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:32.052581 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:32.064341 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1469/2934] 10.222.0.44 () {58 vars in 1153 bytes} [Tue Oct 4 09:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 09:10:32.068649 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2422ed-7d9f-4b39-be2b-ebadce1c7cf4 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:32.077736 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2422ed-7d9f-4b39-be2b-ebadce1c7cf4 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:32.082709 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:32.083301 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.083576 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.083850 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.084503 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:32.086394 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:32.118222 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1466/2935] 10.222.0.44 () {60 vars in 1153 bytes} [Tue Oct 4 09:10:32 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:10:32.122909 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:32.135455 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81e296-22df-4cf7-ab95-7720ad8d87d3 admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:32.194644 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:32.195241 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.195492 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.195759 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] PATH_INFO: `/v3/users` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.196449 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:32.198355 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:32.228310 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1470/2936] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Oct 4 09:10:32 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:10:32.234406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:32.248089 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a690f-e52b-4564-9af0-0baa68ac6d2d admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:32.253753 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:32.254447 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.254755 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.255023 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] PATH_INFO: `/v3/roles` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.255674 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:32.257317 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:32.267835 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1467/2937] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:10:32.272193 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:32.282245 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587bd6-cb13-4c44-a0ee-5c1c4a53c57d admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:32.287160 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf21778-8ef8-4dba-9054-4be5b781c7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:32.287689 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf21778-8ef8-4dba-9054-4be5b781c7fd admin admin] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.288006 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf21778-8ef8-4dba-9054-4be5b781c7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.288300 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf21778-8ef8-4dba-9054-4be5b781c7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.288917 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf21778-8ef8-4dba-9054-4be5b781c7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:32.290494 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf21778-8ef8-4dba-9054-4be5b781c7fd admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:32.301254 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1471/2938] 10.222.0.44 () {58 vars in 1122 bytes} [Tue Oct 4 09:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:10:32.305343 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:32.316636 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fadbb3-068a-4af5-894d-a9748607eb88 admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:32.322837 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:32.323479 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] REQUEST_METHOD: `PUT` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.323751 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.324100 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] PATH_INFO: `/v3/projects/a524f9c10cd343f094a3bd92fad7dd00/users/94c2973267a84932a4a36d26b150dc3d/roles/503f8b8f2dc44b52a8f6901954713e7d` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.341953 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a524f9c10cd343f094a3bd92fad7dd00, user_id=94c2973267a84932a4a36d26b150dc3d, role_id=503f8b8f2dc44b52a8f6901954713e7d)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:32.346281 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:32.363148 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1468/2939] 10.222.0.44 () {60 vars in 1483 bytes} [Tue Oct 4 09:10:32 2022] PUT /identity/v3/projects/a524f9c10cd343f094a3bd92fad7dd00/users/94c2973267a84932a4a36d26b150dc3d/roles/503f8b8f2dc44b52a8f6901954713e7d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:32.379505 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1feb8-528b-406f-9cfb-0602c9ab782a None None] REQUEST_METHOD: `POST` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.379828 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1feb8-528b-406f-9cfb-0602c9ab782a None None] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.380214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1feb8-528b-406f-9cfb-0602c9ab782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.413951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.auth.core [None req-59f1feb8-528b-406f-9cfb-0602c9ab782a None None] MFA Rules not processed for user `94c2973267a84932a4a36d26b150dc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=103427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:32.415152 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-59f1feb8-528b-406f-9cfb-0602c9ab782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:32.463374 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:32.481309 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1472/2940] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:32 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:32.493311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:32.495369 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:32.510688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:32.562235 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17af45-a112-4efb-9035-055d892392cb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:32.566122 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b41fc9-a1a5-4104-9d20-35a00f2c91ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd877daa4c2c14d0c8d281aeebab27ee1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:32.566729 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b41fc9-a1a5-4104-9d20-35a00f2c91ad service cinder] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:32.567003 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b41fc9-a1a5-4104-9d20-35a00f2c91ad service cinder] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:32.567271 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b41fc9-a1a5-4104-9d20-35a00f2c91ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:32.576336 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b41fc9-a1a5-4104-9d20-35a00f2c91ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:32.577773 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b41fc9-a1a5-4104-9d20-35a00f2c91ad service cinder] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:32.589535 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1469/2941] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:32 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:40.801721 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1feb8-528b-406f-9cfb-0602c9ab782a None None] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:40.869062 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1feb8-528b-406f-9cfb-0602c9ab782a None None] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:40.875583 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:40.876208 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] REQUEST_METHOD: `DELETE` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:40.876481 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:40.876744 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] PATH_INFO: `/v3/users/94c2973267a84932a4a36d26b150dc3d` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:40.877995 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c2973267a84932a4a36d26b150dc3d)` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:40.879930 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:40.937920 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c2973267a84932a4a36d26b150dc3d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:40.938592 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c2973267a84932a4a36d26b150dc3d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:40.939204 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c2973267a84932a4a36d26b150dc3d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:40.946671 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c2973267a84932a4a36d26b150dc3d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:40.960763 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.notifications [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c2973267a84932a4a36d26b150dc3d'} {{(pid=103427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:40.976153 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1473/2942] 10.222.0.44 () {58 vars in 1224 bytes} [Tue Oct 4 09:10:40 2022] DELETE /identity/v3/users/94c2973267a84932a4a36d26b150dc3d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:40.980963 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] REQUEST_METHOD: `POST` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:40.981461 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:40.981585 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:41.011311 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.auth.core [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] MFA Rules not processed for user `6a7ff5148f6b41108f35cb9aa85b66e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:10:41.012644 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:41.074145 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1470/2943] 10.222.0.44 () {58 vars in 958 bytes} [Tue Oct 4 09:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:10:41.084196 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:41.085869 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:41.145330 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c062cd2-f570-42ad-8d9c-e155af7027af admin admin] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:41.150059 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ee49c297749468e09c4aa3dc9c218', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:41.150668 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:41.150948 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:41.151214 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:41.160893 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:41.162311 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:41.171683 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1474/2944] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:41.463114 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:41.464629 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:41.478728 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef627d0a-9328-4ebc-9b33-5ad84ce25bd5 None None] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:41.485406 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7ff5148f6b41108f35cb9aa85b66e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d6b22621efa4e2c8d25ad2792cb5242', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:41.486041 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:41.486192 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:41.486455 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] PATH_INFO: `/v3/projects/a524f9c10cd343f094a3bd92fad7dd00` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:41.487688 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a524f9c10cd343f094a3bd92fad7dd00)` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:41.489995 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:41.545604 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a524f9c10cd343f094a3bd92fad7dd00', 'request_id': 'req-770c376f-149d-4cdc-b659-ff884741a6bb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:41.551873 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a524f9c10cd343f094a3bd92fad7dd00', 'request_id': 'req-770c376f-149d-4cdc-b659-ff884741a6bb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:41.557972 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a524f9c10cd343f094a3bd92fad7dd00', 'request_id': 'req-770c376f-149d-4cdc-b659-ff884741a6bb'} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:41.572290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] The token cache is being invalidate because project a524f9c10cd343f094a3bd92fad7dd00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:10:41.572572 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:41.573290 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.notifications [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:10:41.574877 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1471/2945] 10.222.0.44 () {58 vars in 1233 bytes} [Tue Oct 4 09:10:41 2022] DELETE /identity/v3/projects/a524f9c10cd343f094a3bd92fad7dd00 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:10:51.167951 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:51.569350 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:54.180799 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] Authenticating user token {{(pid=103427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:54.182483 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:54.282850 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21160c7-7ebf-4773-a460-20affd746667 service neutron] Validating token access rules against request {{(pid=103427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:54.353489 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb74a3f-2165-4835-8b85-16007b9d467a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:54.354091 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb74a3f-2165-4835-8b85-16007b9d467a service placement] REQUEST_METHOD: `GET` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:54.354360 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb74a3f-2165-4835-8b85-16007b9d467a service placement] SCRIPT_NAME: `/identity` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:54.354630 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb74a3f-2165-4835-8b85-16007b9d467a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:54.356640 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.fernet_utils [None req-0cb74a3f-2165-4835-8b85-16007b9d467a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:54.369103 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb74a3f-2165-4835-8b85-16007b9d467a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:54.370659 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb74a3f-2165-4835-8b85-16007b9d467a service placement] RBAC: Authorization granted {{(pid=103427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:54.380752 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: [pid: 103427|app: 0|req: 1475/2946] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:11:04.376620 dv-cinder-844563-14-iscsi devstack@keystone.service[103427]: DEBUG dbcounter [-] [103427] Writing DB stats keystone:SELECT=67 {{(pid=103427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:15:58.182318 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Authenticating user token {{(pid=103426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:15:58.183924 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.fernet_utils [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:15:58.199686 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c376f-149d-4cdc-b659-ff884741a6bb admin admin] Validating token access rules against request {{(pid=103426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:15:58.205378 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5cce6-8e08-4a0c-a901-ae31c89aceb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cee970f95754740b2f19e64f4653c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c88e95e15cef4d41976389c691f4bca4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:15:58.205985 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cce6-8e08-4a0c-a901-ae31c89aceb6 service placement] REQUEST_METHOD: `GET` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:15:58.206264 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cce6-8e08-4a0c-a901-ae31c89aceb6 service placement] SCRIPT_NAME: `/identity` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:15:58.206574 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5cce6-8e08-4a0c-a901-ae31c89aceb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:15:58.215819 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5cce6-8e08-4a0c-a901-ae31c89aceb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:15:58.217445 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5cce6-8e08-4a0c-a901-ae31c89aceb6 service placement] RBAC: Authorization granted {{(pid=103426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:15:58.227756 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: [pid: 103426|app: 0|req: 1472/2947] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Oct 4 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:16:08.224155 dv-cinder-844563-14-iscsi devstack@keystone.service[103426]: DEBUG dbcounter [-] [103426] Writing DB stats keystone:SELECT=6 {{(pid=103426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}